WO2022027251A1

WO2022027251A1 - Three-dimensional display method and ultrasonic imaging system

Info

Publication number: WO2022027251A1
Application number: PCT/CN2020/106893
Authority: WO
Inventors: 于开欣; 丛龙飞; 王超; 周文兵
Original assignee: 深圳迈瑞生物医疗电子股份有限公司
Priority date: 2020-08-04
Filing date: 2020-08-04
Publication date: 2022-02-10
Also published as: CN115811961A

Abstract

A three-dimensional display method and an ultrasound imaging system. The method comprises: obtaining an ultrasonic image acquired for a lesion, and registering the ultrasonic image with a three-dimensional image comprising the lesion; segmenting the lesion in the three-dimensional image, and displaying a three-dimensional model of the lesion in a three-dimensional display window on the basis of the registration result and the segmentation result; displaying a three-dimensional model of an ablation lesion in the three-dimensional display window according to the registration result and an obtained ablation parameter; determining an ablation zone according to a spatial relation between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion; and if the spatial relation between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfies a rotation condition, automatically rotating the current view from the current angle of view to a target angle of view where the ablation zone meets predetermined requirements of the ablation lesion, and displaying the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion at the target angle of view. The system comprises an ultrasonic probe (110), a transmit circuit (112), a receive circuit (114), a processor (118), and a display (120). According to the three-dimensional display solution, an angle of view for observing a three-dimensional model of a lesion and a three-dimensional model of an ablation lesion can be automatically rotated, thereby facilitating user operations.

Description

Three-dimensional display method and ultrasonic imaging system

technical field

The present application relates to the technical field of ultrasound imaging, and more particularly, to a three-dimensional display method and an ultrasound imaging system.

Background technique

Real-time ultrasound-guided percutaneous tumor ablation intervention has the advantages of high curative effect, less invasion and quick postoperative recovery, and its status in tumor treatment is becoming more and more important. The current interventional treatment of tumor ablation is mainly based on two-dimensional ultrasound image guidance, that is, the doctor finds the approximate location of the tumor area through real-time ultrasound images or contrast-enhanced ultrasound images, and roughly estimates the two-dimensional surface where the largest diameter of the tumor is located. Based on the two-dimensional image Develop an ablation plan and guide the ablation.

With the development of new technologies, it is now possible to use computer 3D reconstruction software and image processing technology to perform 3D reconstruction of the collected 2D images, and to draw target structures such as tumors in the 3D images, thereby realizing the 3D visualization of the target structures. Three-dimensional visualization of stereoscopic structure images can provide areas that are difficult to display in two-dimensional images and obtain objective anatomical information. Based on the real-time ultrasound image and the position information of the ablation needle provided by the positioning device, after the real-time ultrasound image, the ablation needle and the three-dimensional image are registered and mapped to the same space, the three-dimensional visualization display window can objectively and accurately display the tumor, The positional relationship of the ablation needle and the current ablation for the lesion allow the doctor to intuitively plan the tumor operation, optimize the operation plan, improve the operation skills, and thus improve the safety of the operation.

However, during the actual operation, the doctor can only see the ablation situation of the 3D tumor displayed at the current angle when performing tumor ablation, but cannot see the ablation images of the posterior side of the tumor and other angles. After the ablation is complete, it is necessary to manually rotate to find the next angle suitable for ablation, which will cause inconvenience in clinical applications.

SUMMARY OF THE INVENTION

A series of concepts in simplified form have been introduced in the Summary section, which are described in further detail in the Detailed Description section. The summary of the present invention is not meant to attempt to limit the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the protection scope of the claimed technical solution.

An aspect of an embodiment of the present invention provides a three-dimensional display method, the method includes:

obtaining ultrasound images acquired for the lesion;

registering the ultrasound image with a pre-acquired three-dimensional image containing the lesion;

segmenting the lesion in the three-dimensional image containing the lesion, and displaying the three-dimensional model of the lesion in the three-dimensional display window based on the result of the registration and the result of the segmentation;

acquiring ablation parameters of the ablation lesion, and displaying the three-dimensional model of the ablation lesion in the three-dimensional display window according to the registration result and the acquired ablation parameters;

determining the ablation area according to the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion; and

When the spatial relationship between the 3D model of the lesion and the 3D model of the ablation lesion satisfies the rotation condition, the current view of the 3D model of the lesion and the 3D model of the ablation lesion in the 3D display window is changed from the current view The viewing angle is automatically rotated to a target viewing angle in which the ablation area meets the predetermined requirements of the ablation lesion, and the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion under the target viewing angle are displayed.

Another aspect of the embodiments of the present application provides a three-dimensional display method, the method includes:

Display two or more three-dimensional display windows on the display interface;

displaying the current view in one of the two or more three-dimensional display windows, wherein the current view includes the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion under the current viewing angle;

The remaining three-dimensional display windows of the two or more windows display views from other perspectives, and the views from other perspectives include the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion from other perspectives.

Another aspect of the embodiments of the present invention provides an ultrasonic imaging system, the ultrasonic imaging system includes:

Ultrasound probe;

a transmitting circuit, used to excite the ultrasonic probe to transmit ultrasonic waves to the lesion;

a receiving circuit, configured to control the ultrasonic probe to receive the ultrasonic echo returned from the lesion to obtain an ultrasonic echo signal;

processor for:

processing the ultrasonic echo signal to obtain an ultrasonic image;

Segmenting the lesion in the three-dimensional image containing the lesion, and displaying the three-dimensional model of the lesion in the three-dimensional display window of the display based on the result of the registration and the result of the segmentation;

Determine the ablation area according to the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion;

When the rotation condition is met, the current view of the three-dimensional display window is rotated from the current viewing angle to a target viewing angle in which the ablation area meets the predetermined requirements for ablating the lesion, and the three-dimensional model of the lesion and all the lesions in the target viewing angle are displayed. The three-dimensional model of the ablation lesion;

The display is used for displaying the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion.

The three-dimensional display solution of the present application can automatically rotate the viewing angle of the three-dimensional model for observing the lesion and the three-dimensional model of the ablation lesion, which greatly facilitates the ablation operation.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

In the attached image:

1 shows a schematic block diagram of an ultrasound imaging system according to an embodiment of the present invention;

FIG. 2 shows a schematic flowchart of a three-dimensional display method according to an embodiment of the present invention;

3 shows a schematic diagram of spatial transformation in a three-dimensional display method according to an embodiment of the present invention;

4 shows a schematic diagram of a three-dimensional display window in a three-dimensional display method according to an embodiment of the present invention;

5 shows a schematic diagram of a main window and an auxiliary window in a three-dimensional display method according to an embodiment of the present invention;

6 shows a schematic diagram of the interface shown in FIG. 5 after rotating the viewing angle according to an embodiment of the present invention;

FIG. 7 shows a schematic flowchart of a three-dimensional display method according to another embodiment of the present invention;

FIG. 8 shows a schematic diagram of a display interface in a three-dimensional display method according to an embodiment of the present invention.

detailed description

In order to make the objects, technical solutions and advantages of the present application more apparent, the exemplary embodiments according to the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced without one or more of these details. In other instances, some technical features known in the art have not been described in order to avoid confusion with the present application.

It should be understood that the application may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this application to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a," "an," and "the/the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the terms "compose" and/or "include", when used in this specification, identify the presence of stated features, integers, steps, operations, elements and/or components, but do not exclude one or more other The presence or addition of features, integers, steps, operations, elements, parts and/or groups. As used herein, the term "and/or" includes any and all combinations of the associated listed items.

For a thorough understanding of the present application, detailed structures will be presented in the following description in order to explain the technical solutions proposed by the present application. Alternative embodiments of the present application are described in detail below, however, the present application may have other embodiments in addition to these detailed descriptions.

In the following, an ultrasound imaging system according to an embodiment of the present application is first described with reference to FIG. 1 , which shows a schematic structural block diagram of an ultrasound imaging system 100 according to an embodiment of the present application.

As shown in FIG. 1 , the ultrasound imaging system 100 includes an ultrasound probe 110 , a transmitting circuit 112 , a receiving circuit 114 , a processor 118 , and a display 120 . Further, the ultrasound imaging system may further include a beam forming circuit 116 and a transmit/receive selection switch 122 , and the transmit circuit 112 and the reception circuit 114 may be connected to the ultrasound probe 110 through the transmit/receive selection switch 122 .

Wherein, the ultrasonic probe 110 includes an array of multiple transducer array elements, which are used for transmitting ultrasonic waves according to electrical signals, or converting received ultrasonic echoes into electrical signals. Multiple transducers can be arranged in a row to form a linear array, or arranged in a two-dimensional matrix to form an area array, and multiple transducers can also form a convex array, a phased array, etc. The arrangement of the array elements is not limited.

The transducer can transmit ultrasonic waves according to the excitation electrical signal, or convert the received ultrasonic waves into electrical signals, so each transducer can be used to transmit ultrasonic waves to the tissue in the target area, and can also be used to receive ultrasonic echoes returned by the tissue. During ultrasonic measurement, the transmitter circuit 112 and the receiver circuit 114 can control which transducers are used for transmitting ultrasonic waves and which transducers are used for receiving ultrasonic waves, or control the transducers in time slots for transmitting ultrasonic waves or receiving ultrasonic echoes. All transducers participating in ultrasonic emission can be excited by electrical signals at the same time to emit ultrasonic waves at the same time; or transducers participating in ultrasonic emission can also be excited by several electrical signals with a certain time interval, so as to continuously emit ultrasonic waves with a certain time interval .

The ultrasonic probe 110 is provided with a positioning sensor. When the ultrasonic probe 110 moves, the specific position of the current ultrasonic fan can be known according to the coordinate change of the positioning sensor. In some embodiments, a puncture frame is installed on the ultrasound probe 110 for fixing the ablation needle during the ablation procedure, and the angle and position of the ablation needle can be known based on the angle of the puncture frame and the depth of the ablation needle. In some embodiments, a positioning sensor (Vtrax) is provided on the ablation needle, and when the ablation needle moves, the angle and position of the ablation needle can be known according to the coordinate change of the positioning sensor.

During the ultrasound imaging process, the transmit circuit 112 transmits the delayed focused transmit pulses to the ultrasound probe 110 through the transmit/receive selection switch 122 . The ultrasonic probe 110 is stimulated by the transmission pulse to transmit an ultrasonic beam to the target area of the object under test, and after a certain delay, receives the ultrasonic echo with tissue information reflected from the target area, and reconverts the ultrasonic echo into a electric signal. The receiving circuit 114 receives the electrical signals converted and generated by the ultrasonic probe 110, obtains ultrasonic echo signals, and sends these ultrasonic echo signals to the beam forming circuit 116, and the beam forming circuit performs focusing delay, weighting and channel calculation on the ultrasonic echo data. and etc., and then sent to the processor 118. The processor 118 performs signal detection, signal enhancement, data conversion, logarithmic compression and other processing on the ultrasonic echo data to form an ultrasonic image. The ultrasound images obtained by the processor 118 can be displayed on the display 120 or stored in a memory.

Alternatively, the processor 118 may be implemented as software, hardware, firmware, or any combination thereof, and may use single or multiple application specific integrated circuits (ASICs), single or multiple general-purpose integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or any combination of the foregoing circuits and/or devices, or other suitable circuits or devices. Also, the processor 118 may control other components in the ultrasound imaging system 100 to perform corresponding steps of the methods in the various embodiments in this specification.

The display 120 is connected to the processor 118, and the display 120 may be a touch display screen, a liquid crystal display screen, etc.; or the display 120 may be an independent display device such as a liquid crystal display, a television set, etc. independent of the ultrasound imaging system 100; or the display 120 may be Displays of electronic devices such as smartphones, tablets, etc. The number of displays 120 may be one or more. For example, the display 120 may include a main screen and a touch screen, the main screen is mainly used for displaying ultrasound images, and the touch screen is mainly used for human-computer interaction.

Display 120 may display ultrasound images obtained by processor 118 . In addition, the display 120 can also provide a graphical interface for the user to perform human-computer interaction while displaying the ultrasound image, set one or more controlled objects on the graphical interface, and provide the user with a human-computer interaction device to input operating instructions to control these objects. The controlled object, so as to perform the corresponding control operation. For example, an icon is displayed on a graphical interface, and the icon can be operated by using a human-computer interaction device to perform a specific function, such as rotating the angle of view of the current view.

Optionally, the ultrasound imaging system 100 may further include other human-computer interaction devices other than the display 120, which are connected to the processor 118. For example, the processor 118 may be connected to the human-computer interaction device through an external input/output port. / The output port can be a wireless communication module, a wired communication module, or a combination of the two. External input/output ports may also be implemented based on USB, bus protocols such as CAN, and/or wired network protocols, and the like.

Wherein, the human-computer interaction device may include an input device for detecting the user's input information, for example, the input information may be a control instruction for the ultrasonic transmission/reception sequence, or a point, line or frame drawn on the ultrasonic image. Manipulate input instructions, or may also include other instruction types. The input device may include one or a combination of a keyboard, a mouse, a scroll wheel, a trackball, a mobile input device (eg, a mobile device with a touch display screen, a cell phone, etc.), a multi-function knob, and the like. The human-computer interaction apparatus may also include an output device such as a printer.

The ultrasound imaging system 100 may also include memory for storing instructions executed by the processor, storing received ultrasound echoes, storing ultrasound images, and the like. The memory may be a flash memory card, solid state memory, hard disk, or the like. It may be volatile memory and/or non-volatile memory, removable memory and/or non-removable memory, and the like.

It should be understood that the components included in the ultrasound imaging system 100 shown in FIG. 1 are only illustrative, and may include more or less components. This application is not limited to this.

Hereinafter, a three-dimensional display method according to an embodiment of the present application will be described with reference to FIG. 2 . FIG. 2 is a schematic flowchart of a three-dimensional display method 200 according to an embodiment of the present application. The three-dimensional display method 200 in this embodiment of the present application can be used for surgical planning of percutaneous ablation surgery.

As shown in FIG. 2 , the three-dimensional display method 200 according to the embodiment of the present application includes the following steps:

Step S210, acquiring an ultrasound image collected for the lesion.

Wherein, the lesion is the lesion area of the target part of the measured object. The measured object can be a patient who needs to undergo ablation, the target part of the measured object can be the liver, and the lesion area can be the liver tumor area; the target part of the measured object can also be the prostate, thyroid, breast, etc. location, the lesion area is the lesion area of the above-mentioned location. In the following, the three-dimensional display solution of the present application is mainly described by taking the target site as the liver site as an example, but it should be understood that this is only an example, and the three-dimensional display solution of the present application can also be used for any other site.

Exemplarily, referring to FIG. 1 , in step S210 , the ultrasonic probe 110 may be excited by the transmit/receive selection switch 122 to transmit ultrasonic waves to the target part of the measured object via the transmit circuit 112 at regular intervals, and the ultrasonic probe 110 receives the ultrasonic waves via the receive circuit 114 . The ultrasonic echoes returned from the target part of the measured object are converted into ultrasonic echo signals. The beamforming module 116 may perform signal processing, and then send the beamformed ultrasound echo data to the processor 118 for related processing, thereby obtaining an ultrasound image.

According to the different imaging modes required by the user, the processor 118 can perform different processing on the ultrasonic echo signals to obtain ultrasonic data of different modes, and then, through logarithmic compression, dynamic range adjustment, digital scan conversion, etc., to form different modes of ultrasound data. Ultrasound images, such as two-dimensional ultrasound images including B images, C images, and D images.

In practical applications, the operator can use the ultrasound probe to scan the target part of the object to be measured. When the anatomical structure of the lesion appears in the scanned image, the ultrasound image can be frozen to obtain the ultrasound image collected for the lesion.

Step S220, registering the ultrasound image with a pre-acquired three-dimensional image containing the lesion.

As an example, the operator may import the three-dimensional image containing the lesion into the ultrasound imaging system in advance before starting the ultrasound measurement, and the import method includes but is not limited to importing through a storage medium such as a U disk, a CD-ROM, etc. or importing through network transmission.

Three-dimensional images containing lesions can be obtained by computed tomography (CT, Computed Tomography), magnetic resonance imaging (MRI, Magnetic Resonance Imaging), positron emission tomography (PET, Positron Emission Tomography), digital X-ray imaging equipment , ultrasound equipment, digital image subtraction equipment (DSA, Digital Subtraction Angiography), optical imaging equipment and other medical imaging equipment. Because 3D reconstruction takes a long time, 3D images containing lesions are pre-acquired before surgery. The registration of the three-dimensional image collected before operation with the ultrasound image can utilize the spatial information of the three-dimensional image, and at the same time, it has the real-time nature of the ultrasound image.

The registration of the ultrasound image and the three-dimensional image is to seek the spatial transformation relationship between the ultrasound image and the three-dimensional image, so that the geometric relationship between the corresponding points of the ultrasound image and the three-dimensional image is in one-to-one correspondence. Registration may include rigid body registration or non-rigid body registration.

In one embodiment, the positioning sensor fixed on the ultrasound probe can continuously provide position information along with the movement of the ultrasound probe, and the 6-DOF spatial orientation of the ultrasound probe can be obtained through the magnetic positioning controller, and the image information and the magnetic positioning information can be used. The ultrasound image and the three-dimensional image can be registered and fused. The processor may be connected to the positioning sensor provided on the ultrasonic probe through a wired or wireless manner to acquire probe position information. The positioning sensor can use any type of structure or principle, such as an optical positioning sensor or a magnetic field positioning sensor, to position the ultrasonic probe.

The general spatial transformation relationship between ultrasound images and 3D images is shown in Figure 3, which is expressed in the form of a formula as:

X _sec =P·R _probe ·A·X _us

Among them, X _us is the coordinate of the point in the ultrasound image space, X _sec is the coordinate of the corresponding point in the three-dimensional image space; A is the ultrasound image space (the coordinates are expressed as X _us , Yu _us , Z _us ) to the positioning sensor space (the coordinates are expressed as is the transformation matrix of X _sensor , Y _sensor , Z _sensor ), R _probe is the transformation matrix from the positioning sensor space to the world coordinate space (the coordinates are expressed as X _MG , Y _MG , Z _MG ), P is the world coordinate system to the three-dimensional image space transformation matrix. In the ultrasonic measurement process, the positioning sensor is fixed on the ultrasonic probe. When the model of the ultrasonic probe is unchanged, A is fixed, and A can be determined by the method of calibration before registration. The R _probe is directly read by the magnetic positioning controller, and the R _probe changes continuously with the movement of the ultrasonic probe. P is calculated by the result of the registration, that is, the image registration result in the ultrasound space and the three-dimensional image space is M, then:

P=M·A ^-1 ·R _probe ^-1 ; M=P·A·R _probe

The registration methods used in the embodiments of the present application may include automatic registration, interactive registration, manual registration, or any combination of the above three methods. The registration may include registration based on anatomical features or registration based on geometric features, registration based on pixel grayscale correlation, registration based on external localization landmarks, and the like. Registration may also include any other suitable registration method.

In one embodiment, registering the ultrasound image and the three-dimensional image specifically includes: aligning the ultrasound image with a corresponding slice in the three-dimensional image, and determining the ultrasound according to the coordinates of the point in the ultrasound image and the coordinates of the corresponding point in the three-dimensional image. The coordinate transformation matrix from image space to 3D image space. The above-mentioned alignment operation can be performed manually by the user, that is, the user's manual alignment operation is received, so as to align the ultrasound image with the corresponding slice in the three-dimensional image.

In another embodiment, the same tissue in the ultrasound image and the three-dimensional image can be identified for automatic alignment. When the target site is a liver site, the identified same tissue is, for example, a blood vessel, a liver capsule, or the like. After aligning the ultrasound image with the 3D image, a spatial transformation matrix can be calculated from the coordinates of the coincident points.

In other embodiments, the feature points in the ultrasound image and the three-dimensional image may be determined first, and the feature points generally have some of translation invariance, rotation invariance, scale invariance, insensitivity to illumination, insensitivity to modality, etc. The properties of the feature points are determined by the feature point extraction method. Then the features of the feature points are extracted, and the features can be generated by neighborhood gradient histogram, neighborhood autocorrelation, grayscale, etc. After that, the feature points of the ultrasound image are matched with the feature points of the three-dimensional ultrasound image, and a spatial transformation matrix is calculated based on the matched feature points.

In addition, it is also possible to identify the imaging position of the in vitro marker in the three-dimensional image, and determine the position of the in vitro marker in the space of the ultrasound image based on magnetic navigation for automatic alignment. Among them, the in vitro markers are, for example, one or more metal markers set on the body surface of the patient, which will form obvious light spots in the three-dimensional image, and then obtain the position of the marker. In the process of scanning the ultrasonic image, the ultrasonic probe A sensor (sensor) is installed on the sensor, and the position of the metal marker can be obtained through the sensor, and the registration of the ultrasound image and the three-dimensional image can be realized by aligning the marker in the three-dimensional ultrasound image with the marker in the ultrasound image.

When the three-dimensional image of the lesion area is a three-dimensional ultrasound image, if it has its own position information, automatic registration can be performed based on the position information of the ultrasound image and the three-dimensional ultrasound image. The 3D image may be acquired by a volume probe, or reconstructed by a convex array or linear array probe with a magnetic navigation device based on the Freehand 3D ultrasound reconstruction technology, or scanned by an area array probe. The 3D ultrasound image reconstructed based on the magnetic navigation position information can be a reconstructed 3D ultrasound image obtained by scanning an ultrasound movie with positioning information on-site Freehand, and the position information can be obtained during scanning, so the P matrix above can be obtained automatically.

In one embodiment, for the ablation of soft tissues in the abdomen, such as the liver and lungs, due to the influence of the patient's breathing movement, the position of the soft tissue and the lesion is shifted, so a respiration correction function is introduced during the registration process to perform respiration correction. . As shown in Figure 3, the added T(t) is a spatial mapping method for respiration correction, and T(t) changes with time, then the spatial transformation relationship between the ultrasonic image and the three-dimensional image is expressed in the form of a formula as:

X _Sec =T(t)·P·R _probe ·A·X _us

In addition, the position deviation caused by the breathing movement can also be corrected by making the patient breathe smoothly.

In one embodiment, the ultrasonic image and the three-dimensional image may also be fused according to the corresponding relationship between the ultrasonic image and the three-dimensional image obtained in step S220, and the fusion of the ultrasonic image and the three-dimensional image is displayed in the fusion display window of the display image. Specifically, the ultrasound image space coordinate system can be mapped to the three-dimensional image space coordinate system based on the registration relationship matrix. Since the positioning sensor is installed on the ultrasound probe, when the ultrasound probe moves, according to the coordinate change of the positioning sensor, the current ultrasound can be known. The specific positional relationship between the fan and the lesion located in the three-dimensional image space coordinate system.

Step S230, segment the lesion in the three-dimensional image containing the lesion, and display the three-dimensional model of the lesion in the three-dimensional display window based on the registration result and the segmentation result. For example, a three-dimensional display window is set on the display interface of the display for displaying the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion to be described below.

As an example, the 3D contour of the lesion is first segmented in the 3D image. In the embodiments of the present application, any suitable method may be used to segment the lesions in the three-dimensional image, including but not limited to automatic segmentation, manual segmentation, or interactive segmentation. Exemplarily, the automatic segmentation method may adopt one or more of random walk model, region growing, graph cut algorithm, pattern recognition, Markov field, adaptive threshold and other methods. Manual segmentation involves the user delineating the edges of the lesion on multiple 2D slices of the 3D data and interpolating between the edges of each two slices, or outlining the edges of the lesion on each 2D slice, and then based on these 2D slices. The edges generate a three-dimensional outline of the lesion. Interactive segmentation methods add user interaction as algorithm input in the segmentation process, so that objects with high-level semantics in the image can be completely extracted. For example, a preliminary segmentation range can be selected by the user; then, the three-dimensional contour of the lesion is automatically segmented within the preliminary segmentation range. For another example, the user can draw some points or lines within the preliminary segmentation range, obtain the points or lines drawn by the user as input through an interactive segmentation algorithm, and automatically establish a weighted map of the similarity between each pixel and the foreground or background, And by solving the minimum cut to distinguish the foreground and background, the three-dimensional contour of the lesion is determined.

Next, surface reconstruction is performed according to the three-dimensional contour obtained by the segmentation to generate a three-dimensional model of the lesion, such as the three-dimensional model 401 of the lesion as shown in FIG. 4 .

Among them, the surface rendering method can be used to first reconstruct the surface of the 3D target structure from the 3D data, that is, to reconstruct the surface of the object according to the segmentation results and contour lines, and then use a reasonable illumination model and texture mapping method to generate a realistic 3D surface. entity. It can be understood that since the image displayed by the display is still a two-dimensional plane, the surface rendering is actually a projection showing the reality of the three-dimensional object on the two-dimensional plane, similar to when the viewing angle is located at a certain point, The three-dimensional object is "photographed", and the image of the three-dimensional object is displayed on the photo.

Then, according to the coordinates of the three-dimensional contour of the lesion in the three-dimensional image, and according to the registration relationship between the ultrasound image and the three-dimensional image obtained in step S220, display the three-dimensional image of the lesion at a certain position (eg, the middle position) of the three-dimensional display window. model, and based on the registration relationship between the ultrasonic image and the 3D image obtained in step S220, the ultrasonic image can be displayed in the 3D display window, and the 3D model of the lesion can be displayed at the location of the lesion in the ultrasonic image. The positional relationship between the reconstructed 3D model of the lesion and the real-time ultrasound image can reflect the location, size, geometry of the lesion and its relationship with the surrounding tissue. Specifically, after determining the coordinate X _{sec_tumor} of the lesion in the three-dimensional image, the coordinate X _{us_tumor} of the lesion position in the real-time ultrasound image is calculated based on the registration matrix of the three-dimensional image and the ultrasound image, and the three-dimensional model of the lesion is superimposed and displayed in the three-dimensional display window. the corresponding location. in,

X _{us_tumor} =M ^-1 ·X _{sec_tumor} =(P·R _probe ·A) ^-1 X _{sec_tumor} .

In step S240, the ablation parameters of the ablation lesion are acquired, and the 3D model of the ablation lesion is displayed at the corresponding position in the 3D display window according to the acquired ablation parameters and the registration result obtained in step S220, and the corresponding position is based on the registration result. and the location of the ablation focus determined by the acquired ablation parameters. This embodiment of the present application does not limit the execution order of step S230 and step S240. Through these two steps, the 3D model of the lesion and the 3D model of the ablation lesion can be displayed in the same image space, so that the positional relationship between the lesion and the ablation lesion in the same image space can be easily displayed to the user. According to the registration result, the image space of the three-dimensional display window can be the image space corresponding to the three-dimensional image, the image space corresponding to the ultrasound image, or any other image space, as long as the three-dimensional model of the lesion and the ablation focus are ensured The 3D model can be displayed in the same image space. In the example of this application, two three-dimensional models are displayed in the image space of the three-dimensional image as an example for description.

Three-dimensional models of ablation lesions include single-needle ablation models or multi-needle combined ablation models. The single-needle ablation model is a three-dimensional model of the ablation lesion displayed in the three-dimensional display window according to the single ablation needle inserted into the lesion based on the above-mentioned registration result and the acquired ablation parameters. The multi-needle combined ablation model is a three-dimensional model of the ablation lesion displayed in the three-dimensional display window according to at least two ablation needles inserted into the lesion sequentially or simultaneously based on the above-mentioned registration results and the acquired ablation parameters. For example, when the shape of the ablation lesion is an ellipsoid, the single-needle ablation model of the ablation lesion displayed in the three-dimensional display window is a single ellipsoid, and the multi-needle combined ablation model includes multiple ellipsoids. As an example, the three-dimensional model of the ablation lesion may be displayed in a different color from the three-dimensional model of the lesion to facilitate the distinction between the two. In some embodiments, both the three-dimensional model of the ablation lesion and the three-dimensional model of the lesion can be displayed as translucent figures, so as to facilitate the observation of the overlapping areas of the two. FIG. 4 shows a three-dimensional model 402 of an ablation lesion in one embodiment.

According to the 3D model of the ablation lesion displayed on the display interface and the above-mentioned 3D model of the lesion, the operator can intuitively determine the ablation area, that is, the overlapping area of the 3D model of the ablation lesion and the 3D model of the lesion. As an example, the overlapping area may be displayed in a different color than the three-dimensional model of the ablation lesion and the three-dimensional model of the lesion.

Wherein, for example, the coordinates of the center of the ablation focus in the ultrasound image may be first determined, and the coordinates of the center of the ablation focus in the three-dimensional image may be determined according to the coordinate transformation matrix from the ultrasound image space to the three-dimensional image space; then, according to the position of the center of the ablation focus in the three-dimensional image The coordinates in the three-dimensional image and the size of the ablation lesion, and a three-dimensional model of the ablation lesion is drawn in the image space of the three-dimensional image. When a puncture frame is installed on the ultrasonic probe, the coordinates of the center of the ablation focus in the ultrasonic image can be determined according to the angle of the puncture frame and the depth of the ablation path. When the three-dimensional model of the ablation lesion is a multi-needle combined ablation model, the coordinates of the center of the ablation lesion may be set separately for each of the ablation models.

In addition to the center coordinates of the ablation focus, the ultrasound imaging system also needs to obtain some ablation parameters set by the operator. For example, the operator also needs to set the power of the ablation needle and the continuous ablation time, and obtain the ablation range of the ablation needle to ensure that the ablation area contains A 3D model of the entire lesion and its safe boundaries. The safety margin refers to that during the ablation process, the ablation lesion is generally required to cover the edge of the lesion and expand outward by a certain distance to ensure complete ablation of the entire lesion.

In this step, the operator can input the given power of the ablation operation and the ablation duration, and obtain the size of the ablation area according to the above-mentioned working parameters. The operator may also first set the required ablation area range, that is, a preset ablation area, and select the corresponding given power and ablation duration and other working parameters according to the set ablation area range. Since the ablation range of the ablation needle is usually an ellipsoid, in this step, when setting the range of the ablation region, it is only necessary to set the length of the long axis and the length of the short axis of the ellipsoid. It should be noted that the shape of the ablation focus is not limited to an ellipsoid, but can also include a sphere or a cylinder. The operator can set the shape of the ablation focus according to the shape of the focus, and set different parameters according to the shape of the ablation focus.

The drawing method of the three-dimensional model of the ablation lesion can also be surface drawing. Assuming that the shape of the ablation focus is ellipsoid, the operator can set the long diameter, short diameter, needle tip distance, path depth, etc. of the ablation focus according to the actual ablation needle model. The system uses these parameters to draw the ablation focus at the coordinate origin. , where the needle tip distance is the distance between the heat source of the ablation needle and the needle tip, and the center point of the ablation focus is the position where the depth of the ablation needle punctured path minus the needle tip distance is located. If the puncture of the ablation needle is performed based on the puncture frame set on the ultrasonic probe, the puncture frame angle β needs to be set first, and the position of the ablation focus in the current ultrasonic sector _{Tus_ablate} is calculated using the set puncture frame angle β and the path depth d , i.e. (x _{us_ablate,} y _{us_ablate} ), where:

x _{us_ablate} = d sinβ

y _{us_ablate} = d·cosβ

In another embodiment, the coordinates of the center of the ablation focus in the ultrasound image may be determined according to a positioning sensor disposed on the ablation needle.

Specifically, when the operator performs puncture based on the positioning sensor installed on the ablation needle, the angle and depth of the ablation lesion are obtained based on the coordinate changes of the positioning sensor, that is, the coordinates of the ablation lesion in the positioning sensor space are determined according to the positioning sensor. , according to the transformation matrix from the positioning sensor to the ultrasonic image space, it is converted to the ultrasonic image space, and finally the coordinates of the ablation focus are mapped to the three-dimensional image space according to the registration matrix of the ultrasonic image space and the three-dimensional image space to realize three-dimensional visualization. Assuming that the coordinates of the ablation focus in the ultrasound space are _{Tus_ablate} , then in the three-dimensional image space, its coordinates are:

T _{sec_ablate} =M· _{Tus_ablate} =P·R _probe ·A· _{Tus_ablate}

When a puncture frame is installed on the ultrasonic probe, the operator moves the ultrasonic probe installed with the positioning sensor, and the position of the ablation focus in the three-dimensional image space will move with the change of R _probe in the mapping relationship matrix. As an example, if the operator is satisfied with the current position of the ablation focus, he may click to save, that is, it is considered that the current position of the ablation focus has been ablated.

In one embodiment, two or more three-dimensional display windows may be displayed on the display interface, wherein one window displays the current view, and the other windows display views from other perspectives, so that the operator can simultaneously view the images from multiple perspectives. Lesions and ablation foci were observed.

Specifically, referring to FIG. 5 , the above two or more three-dimensional display windows include a main window 501 and an auxiliary window 502 , where the main window 501 is used to display the current view, and the auxiliary window 502 is used to display views from other perspectives. In some embodiments, on the premise that the 3D model of the lesion and the 3D model of the ablation lesion are not blocked, the auxiliary window 502 can be superimposed and displayed on the corner of the main window 501 as shown in FIG. 5 to save layout. Of course, the auxiliary window 502 can also be displayed side by side with the main window 501 . Generally speaking, the size of the main window 501 is larger than the size of the auxiliary window 502, but not limited to this, and the sizes of the two may also be the same.

As an example, the other viewing angles displayed by the at least one auxiliary window include a reverse viewing angle of the current viewing angle displayed by the main window. For example, the main window 501 in FIG. 5 displays the perspective of the front side of the lesion, and the auxiliary window 502 displays the perspective of the rear side of the lesion. In addition, auxiliary windows can also be set to display other perspectives such as the left, right, and bottom sides of the lesion, so that the operator can more comprehensively understand the ablation conditions for different angles of the lesion. As an example, the angle relationship between the main window and the auxiliary window may be preset, and the angle of view of the auxiliary window is determined according to the angle of view of the main window and the angle relationship. For example, the preset angular relationship between the main window and the auxiliary window is an angle difference of 30° in the clockwise direction, and the auxiliary window relative to the main window displays a view from a viewing angle rotated 30° clockwise. As an example, other viewing angles displayed in the auxiliary window are the target viewing angles described below. When there are multiple target viewing angles and one auxiliary window, the auxiliary window may display the view at the viewing angle with the largest unablated area, or display the rotated view. The view from the minimum angle of view. When there are multiple target viewing angles and multiple auxiliary windows, the corresponding views under each viewing angle may be sequentially displayed in the multiple auxiliary windows according to the size of the unablated region or the size of the rotation angle.

Step S250: Determine the ablation area according to the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion.

As an implementation manner, the ablation area is the overlapping area of the 3D model of the lesion and the 3D model of the ablation lesion. Based on the above registration results, the 3D model of the lesion and the 3D model of the ablation lesion can be mapped to the same image space, and the 3D model of the lesion can be calculated. The overlapping area of the model and the three-dimensional model of the ablation lesion is determined as the ablation area. Specifically, in the same image space, it can be determined whether each coordinate point of the three-dimensional model of the lesion falls within the range of the three-dimensional model of the ablation focus, and the position of the lesion that falls within the range of the three-dimensional model of the ablation focus is considered to have been ablated , the lesion location outside the range of the 3D model of the ablation focus is considered not to be ablated, and the set of coordinate points that fall within the range of the 3D model of the ablation focus is determined as the ablated area, and the coordinates are connected through traversal and calculation. Domain and other methods can calculate the ablated coordinate area {P1, P2, ...} and the non-ablated coordinate area {N1, N2, ...} of the lesion. Since the z-axis size of the tumor coordinates displayed in the current view in the model drawing is different from that in other views, the z-axis size of the current face in the current view can be determined based on the z-axis size of the ablated coordinate area and the unablated coordinate area. Ablation ratio and maximum ablation connected area. It can be understood that, in other embodiments, since the ablation area is the overlapping area of the 3D model of the lesion and the 3D model of the ablation lesion, based on similar principles, it is also possible to determine whether each coordinate point of the 3D model of the ablation lesion is not. The ablation area is determined within the confines of the 3D model of the lesion.

Step S260, when the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfies the rotation condition, display the current view of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion in the three-dimensional display window from the current perspective. Automatically rotate to a target viewing angle where the ablation area meets the predetermined requirements of the ablation lesion, and display the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion under the target viewing angle.

In one embodiment, the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfying the rotation condition includes: in the three-dimensional display window, the spatial position of the three-dimensional model of the lesion and the position of the three-dimensional model of the ablation lesion in any viewing angle The inclusion of the spatial location satisfies the preset condition. For example, by judging whether the spatial position of the 3D model of the lesion and the spatial position of the 3D model of the ablation lesion in any viewing angle reach a certain proportion, when the spatial position of the 3D model of the lesion and the 3D model of the ablation lesion in other perspectives When the included part of the spatial position of , reaches a certain proportion, it is judged that the rotation condition is satisfied. Alternatively, it can be judged whether the rotation condition is satisfied by judging whether the spatial position of the 3D model of the lesion and the spatial position of the 3D model of the ablation lesion at any viewing angle are mutually included, for example, if the spatial position of the 3D model of the lesion at other viewing angles is the same as If the spatial positions of the three-dimensional models of the ablation lesions do not include each other, it is determined that the rotation condition is satisfied. Alternatively, it can be determined whether there is a target of interest in the mutually included part of the spatial position of the 3D model of the lesion and the spatial position of the 3D model of the ablation lesion at any viewing angle. It is judged that the rotation condition is satisfied.

In one embodiment, whether the rotation condition is satisfied is determined according to the projected area of the ablation region under the current viewing angle. The projection area mentioned herein refers to the display area of the display view of the 3D model displayed on the display window of the display at the current viewing angle. As mentioned above, since the image displayed on the monitor is a two-dimensional plane observed at a certain angle of view, most of the lesions may have been ablated at the current angle of view, while there are still larger unablated areas at other angles of view. Continuing to observe at the current viewing angle will be detrimental to the subsequent ablation operation, and it will be difficult to determine whether the lesion has been ablated. Therefore, when most of the lesion area has been ablated at the current angle of view, it can be determined that the rotation condition is satisfied, thereby triggering automatic rotation to rotate to a larger angle of view of the non-ablated area of the lesion, so as to continue the ablation.

Among them, it can be determined whether most of the lesions in the current perspective have been ablated by comparing the projected area of the ablation area with the projected area of the 3D model of the lesion at the current perspective, that is, if the projected area of the ablation area is within the 3D model of the lesion If the proportion of the projected area of the model exceeds a predetermined threshold, it is determined that the rotation condition is satisfied.

In addition, since the actual ablation area is the overlapping area of the lesion and the ablation lesion, the projected area of the ablation area at the current viewing angle can also be compared with the projected area of the 3D model of the ablation lesion to determine the size of the lesion at the current viewing angle. Whether most of the regions have been ablated, that is, if the proportion of the projected area of the ablation region in the projected area of the three-dimensional model of the ablation lesion exceeds a predetermined threshold, it is determined that the rotation condition is satisfied.

In some cases, after performing at least one ablation, that is, after generating a 3D model of the ablation lesion, the ablation area seen in the current viewing angle is still too small, or the ablation area cannot be seen in the current viewing angle, indicating that the ablation lesion is The overlapping area with the lesion area may be located in the opposite direction of the current viewing angle or in other different directions, so at this time, the viewing angle of the current view needs to be rotated to better observe the ablation area. Specifically, after the three-dimensional model of at least one ablation lesion is displayed, the projected area of the ablation area in the projected area of the three-dimensional model of the lesion is smaller than a predetermined threshold, or the projected area of the ablation area occupies the projected area of the three-dimensional model of the ablation lesion If the ratio is smaller than the predetermined threshold, it is determined that the rotation condition is satisfied.

In some embodiments, the user can also determine whether the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfies the rotation condition. When the user determines that the viewing angle needs to be rotated, the rotation of the viewing angle can be triggered by manually clicking the automatic rotation function key.

After it is determined that the rotation condition is satisfied, a rotation operation is performed to rotate the current view to a target viewing angle in which the ablation area meets the predetermined requirements of the ablation lesion, so as to better observe the ablation effect. The target viewing angle at which the ablation area meets the predetermined requirements of the ablation lesion may be a viewing angle that is beneficial for the user to observe the ablation area to ablate the lesion.

The target viewing angle may be a viewing angle with a smaller projected area of the ablation region. As an implementation, the projected area of the ablation area can be compared with the projected area of the 3D model of the lesion, that is, the target viewing angle is the perspective with the smallest ratio of the projected area of the ablation area to the projected area of the 3D model of the lesion, that is, the current view Rotate to the target view with the largest unablated area. Alternatively, at least one viewing angle whose ratio between the projected area of the ablation area and the projected area of the three-dimensional model of the lesion can be determined is smaller than or equal to a predetermined threshold, and when there is one viewing angle, the viewing angle is regarded as the target viewing angle with the largest unablated area, and there are more than one viewing angle. When there is one viewing angle, one viewing angle among the more than one viewing angle can be selected as the target viewing angle with the largest unablated area, for example, the viewing angle with the smallest required rotation angle can be selected.

As another implementation, the projected area of the ablation area can also be compared with the projected area of the 3D model of the ablation lesion to find a viewing angle with a smaller projected area of the ablation area, that is, the target viewing angle is the projected area of the ablation area and the The viewing angle with the smallest ratio of the projected area of the three-dimensional model of the ablation lesion, or the target viewing angle is the viewing angle where the ratio of the projected area of the ablation region to the projected area of the three-dimensional model of the ablation lesion is less than or equal to a predetermined threshold.

After the target viewing angle is determined, the angle between the current viewing angle and the target viewing angle is calculated as the rotation angle of the view, and the viewing angle of the view is rotated according to the calculated rotation angle.

Specifically, the coordinates of the projection of the unablated region of the three-dimensional model of the lesion in the current view can be obtained by traversing the coordinate depth of the three-dimensional model of the lesion, and then the center position of the projection of the unablated region of the lesion displayed in the current view can be calculated. The first coordinates of (x ₁ , y ₁ ). Based on a similar method, the second coordinate (x ₂ , y ₂ ) of the center position of the projection of the unablated area of the lesion under the target viewing angle is calculated, and the sum of the connecting line between the first coordinate (x ₁ , y ₁ ) and the coordinate origin is calculated. The included angle α between the second coordinates (x ₂ , y ₂ ), and the included angle α is used as the rotation angle, where:

tanα=(x ₁ y ₂ -x ₂ y ₁ )/(x ₁ x ₂ +y ₁ y ₂ )

α=tan(arctanα)

In the above method, the line between the coordinate origin and the center point of the unablated area is used as the starting and ending point of rotation. As another method, the connecting line between the coordinate origin and the center point of the lesion projection can also be used as the starting and ending point of rotation. . Specifically, determining the third coordinate of the center position of the projection of the three-dimensional model of the lesion under the current viewing angle, and determining the second coordinate of the center position of the projection of the unablated region of the three-dimensional model of the lesion under the target viewing angle; calculating the third coordinate The included angle between the connection line with the coordinate origin and the connection line between the second coordinate and the coordinate origin, and the included angle is used as the rotation angle of the rotation.

In practical applications, there may be multiple viewing angles to meet the predetermined requirements. When there are multiple viewing angles that meet the predetermined requirements, the viewing angle with the smallest required rotation angle may be determined as the target viewing angle.

Referring to the above, in some embodiments, two or more three-dimensional display windows are displayed on the display interface, and one window (for example, the main window) displays the current view under the current viewing angle; then in an example, When the current view is rotated, the views in other windows are rotated synchronously. For example, as shown in FIG. 5 , the main window 501 displays the perspective of the anterior side of the lesion, and the auxiliary window displays the perspective of the rear of the lesion. If the perspective in the main window 501 is rotated to the rear of the lesion, the auxiliary window The viewing angle in 502 will be rotated to the anterior side of the lesion, as shown in FIG. 6 . If the viewing angle in the main window 501 is rotated to the left side of the lesion, the viewing angle in the auxiliary window 502 will be rotated to the right side of the lesion accordingly.

In other examples, when the current view is rotated, the views from other perspectives can also be fixed. For example, if the viewing angle in the main window 501 is rotated to the left side of the lesion, the viewing angle in the auxiliary window can continue to remain at the back side of the lesion.

After that, steps S240 to S260 may be repeatedly performed to continue generating ablation lesions based on the rotated view until the lesions are completely ablated.

To sum up, the three-dimensional display method 200 according to the embodiment of the present invention can automatically rotate the viewing angle for observing the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion, which greatly facilitates the ablation operation.

Referring to FIG. 7, another embodiment of the present application provides a three-dimensional display method 700, which includes:

In step S710, two or more three-dimensional display windows are displayed on the display interface;

In step S720, a current view is displayed in one of the two or more three-dimensional display windows, wherein the current view includes a three-dimensional model of the lesion and a three-dimensional model of the ablation lesion under the current viewing angle;

In step S730, the remaining windows of the two or more windows display views from other perspectives, and the views from the other perspectives include the three-dimensional model of the lesion and the ablation lesion from the other perspectives. 3D model.

In one embodiment, the current view further includes an ultrasound image, and the ultrasound image, the three-dimensional model of the lesion, and the three-dimensional model of the ablation lesion in the same three-dimensional display window are in the same image space, for example, may be in the image space of the ultrasound image or within the image space of a 3D image. Similarly, the views from other viewing angles may also include ultrasound images, and the three-dimensional models of the lesions, the three-dimensional models of ablation lesions and the ultrasound images from other viewing angles are also in the same image space.

Further, referring to FIG. 8 , FIG. 8 shows a schematic diagram of a display interface 800 of the three-dimensional display method 700 according to an embodiment of the present invention. In this embodiment, in addition to the first three-dimensional display window 801 displaying the current view and the second three-dimensional display window 802 displaying views from other perspectives on the display interface 800, the ultrasound window 803 and the fusion display window 804 are also displayed, so The ultrasound window 803 is used to display an ultrasound image, for example, a real-time ultrasound image, and the fusion display window 804 is used to display a fusion image of the ultrasound image and the three-dimensional image including the lesion. In addition to the layout shown in FIG. 8 , other suitable layouts may also be used for the first three-dimensional display window 801 , the second three-dimensional display window 802 , the ultrasound window 803 and the fusion display window 804 .

The three-dimensional display method 700 according to the embodiment of the present invention adopts a multi-viewing angle display manner of two or more windows, which is beneficial for the user to better know the ablation conditions of different viewing angles.

Referring now to FIG. 1 again, an embodiment of the present invention further provides an ultrasound imaging system 100 , and the ultrasound imaging system 100 can be used to implement the above-mentioned three-dimensional display method 200 . The ultrasound imaging system 100 may include an ultrasound probe 110, a transmitting circuit 112, a receiving circuit 114, a beamforming circuit 116, a processor 118, a display 120, a transmit/receive selection switch 122, and some or all of the components in the memory 124. The description can refer to the above. Only the main functions of the ultrasound imaging system 100 are described below, and the details that have been described above are omitted.

Specifically, the transmitting circuit 112 is used to excite the ultrasonic probe 110 to transmit ultrasonic waves to the lesion; the receiving circuit 114 is used to control the ultrasonic probe to receive the ultrasonic echo returned from the lesion to obtain an ultrasonic echo signal; the processor 118 is used to: detect the ultrasonic echo processing the signal to obtain an ultrasound image; registering the ultrasound image with a pre-acquired three-dimensional image containing the lesion; segmenting the lesion in the three-dimensional image containing the lesion, based on the result of the registration With the result of the segmentation, the three-dimensional model of the lesion is displayed in the three-dimensional display window of the display 120; the ablation parameters of the ablation lesion are acquired, and the ablation lesion is displayed in the three-dimensional display window according to the registration result and the acquired ablation parameters The ablation area is determined according to the spatial relationship between the 3D model of the lesion and the 3D model of the ablation lesion; when the rotation conditions are met, the current view of the 3D display window is automatically rotated from the current viewing angle to the ablation area The region meets the target viewing angle of the predetermined requirement of the ablation lesion, and displays the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion under the target viewing angle.

In one embodiment, the ablation area is an overlapping area of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion.

In one embodiment, the satisfying the rotation condition includes: the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfies the rotation condition, or receiving a rotation instruction input by a user.

In one embodiment, the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfying a rotation condition includes: in the three-dimensional display window, the spatial position of the three-dimensional model of the lesion at any viewing angle and The inclusion of the spatial position of the three-dimensional model of the ablation focus satisfies a preset condition.

In one embodiment, the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfies a rotation condition, including: in the current viewing angle, the projected area of the ablation region in the three-dimensional display window The projected area of the three-dimensional model of the lesion or the projected area of the ablation lesion satisfies a preset condition.

In one embodiment, satisfying a preset condition includes: the proportion of the projected area of the ablation area in the projected area of the three-dimensional model of the lesion exceeds a predetermined threshold, or the projected area of the ablation area is within The proportion of the projected area of the three-dimensional model of the ablation lesion exceeds a predetermined threshold.

In one embodiment, the target viewing angle of the ablation area that meets the predetermined requirements of the ablation lesion includes: a viewing angle at which the ratio of the projected area of the ablation area to the projected area of the three-dimensional model of the lesion is the smallest, or the angle of view of the ablation area. The viewing angle at which the ratio of the projected area to the projected area of the 3D model of the lesion is lower than a predetermined threshold; The ratio of the projected area of the ablation region to the projected area of the three-dimensional model of the ablation lesion is lower than the viewing angle of the predetermined threshold.

In one embodiment, rotating the current view from the current viewing angle to a target viewing angle where the projection of the ablation area meets a predetermined requirement includes: determining the first coordinates of the center position of the unablated area of the three-dimensional model of the lesion under the current viewing angle; determining the target viewing angle under the The second coordinate of the center position of the unablated area of the three-dimensional model of the lesion; determine the angle between the line connecting the first coordinate and the coordinate origin and the line connecting the second coordinate and the coordinate origin, and use the angle as the rotation angle of the rotation .

In one embodiment, rotating the current view from the current viewing angle to a target viewing angle where the projection of the ablation area meets a predetermined requirement includes: determining the third coordinate of the center position of the three-dimensional model of the lesion under the current viewing angle; determining the three-dimensional image of the lesion under the target viewing angle The second coordinate of the center position of the unablated region of the model; the included angle between the line connecting the third coordinate and the coordinate origin and the line connecting the second coordinate and the coordinate origin is determined, and the included angle is used as the rotation angle of the rotation.

In one embodiment, the processor 118 is further configured to display two or more three-dimensional display windows on the display interface of the display 120, wherein one window displays the current view, and the other windows display views from other viewing angles.

Exemplarily, the two or more windows include a main window and an auxiliary window, wherein the main window is used to display the current view, and the auxiliary window is used to display views from other perspectives, and the size of the main window may be larger than that of the auxiliary window.

Exemplarily, when the current view is rotated, the views in other perspectives are fixed, or the views in other perspectives can also be rotated synchronously.

In addition, according to the embodiments of the present application, a computer storage medium is also provided, where program instructions are stored on the computer storage medium, and when the program instructions are run by a computer or a processor, the program instructions are used to execute the three-dimensional images of the embodiments of the present application. The corresponding steps of method 200 are displayed. The storage medium may include, for example, a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, or any combination of the above storage media. The computer-readable storage medium can be any combination of one or more computer-readable storage media.

In addition, according to the embodiments of the present application, a computer program is also provided, and the computer program can be stored in the cloud or on a local storage medium. When the computer program is run by a computer or a processor, it is used to execute the corresponding steps of the three-dimensional display method of the embodiments of the present application.

Based on the above description, the three-dimensional display method and the ultrasound imaging system according to the embodiments of the present application can automatically rotate the viewing angle for observing the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion, which greatly facilitates the surgeon's surgical operation.

Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above-described example embodiments are exemplary only, and are not intended to limit the scope of the application thereto. Various changes and modifications may be made therein by those of ordinary skill in the art without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or May be integrated into another device, or some features may be omitted, or not implemented.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that the embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, FIG. , or in its description. However, this method of application should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as the corresponding claims reflect, the invention lies in the fact that the corresponding technical problem may be solved with less than all features of a single disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this application.

It will be understood by those skilled in the art that all features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method or apparatus so disclosed may be used in any combination, except that the features are mutually exclusive. Processes or units are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the present application within and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The present application can also be implemented as a program of apparatus (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present application may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

It should be noted that the above-described embodiments illustrate rather than limit the application, and alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.

The above are only specific embodiments of the present application or descriptions of the specific embodiments, and the protection scope of the present application is not limited thereto. Any changes or substitutions should be included within the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A three-dimensional display method, characterized in that the method comprises:

obtaining ultrasound images acquired for the lesion;

registering the ultrasound image with a pre-acquired three-dimensional image containing the lesion;

Segmenting the lesion in the three-dimensional image containing the lesion, and displaying the three-dimensional model of the lesion in the three-dimensional display window according to the result of the registration and the result of the segmentation;

acquiring ablation parameters of the ablation lesion, and displaying the three-dimensional model of the ablation lesion in the three-dimensional display window according to the registration result and the acquired ablation parameters;

determining the ablation area according to the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion; and

When the spatial relationship between the 3D model of the lesion and the 3D model of the ablation lesion satisfies the rotation condition, the current view of the 3D model of the lesion and the 3D model of the ablation lesion in the 3D display window is changed from the current view The viewing angle is automatically rotated to a target viewing angle in which the ablation area meets the predetermined requirements for ablating the lesion, and the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion under the target viewing angle are displayed.
The method according to claim 1, wherein the ablation area is an overlapping area of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion.
The method of claim 2, wherein determining the ablation region comprises:

Based on the result of the registration, the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion are mapped to the same image space, the overlapping area of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion is calculated, and the overlapping area is calculated. Determine the ablation area.
The method according to claim 3, wherein the calculating the overlapping area of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion comprises:

In the same image space, determine whether each coordinate point of the three-dimensional model of the lesion falls within the range of the three-dimensional model of the ablation focus, and determine the coordinates that fall within the range of the three-dimensional model of the ablation focus The set of points is determined as the ablation area.
The method according to claim 1, wherein the satisfying the rotation condition comprises:

In the three-dimensional display window, the inclusion situation of the spatial position of the three-dimensional model of the lesion and the spatial position of the three-dimensional model of the ablation lesion at any viewing angle satisfies a preset condition.
The method according to claim 1, wherein the satisfying the rotation condition comprises:

Under the current viewing angle, the projected area of the ablation area in the three-dimensional display window and the projected area of the three-dimensional model of the lesion, or the projected area of the ablation area and the projected area of the ablation lesion meet the predetermined requirements Set conditions.
The method according to claim 6, wherein the projected area of the ablation area on the three-dimensional display window and the projected area of the three-dimensional model of the lesion, or the projected area of the ablation area and the The projected area of the ablation lesion meets the preset conditions, including:

The proportion of the projected area of the ablation region to the projected area of the three-dimensional model of the lesion exceeds a predetermined threshold.
The method according to claim 6, wherein the projected area of the ablation area on the three-dimensional display window and the projected area of the three-dimensional model of the lesion, or the projected area of the ablation area and the The projected area of the ablation lesion meets the preset conditions, including:

The proportion of the projected area of the ablation region in the projected area of the three-dimensional model of the ablation lesion exceeds a predetermined threshold.
The method according to claim 1, wherein the three-dimensional model of the ablation lesion is at least one of a single-needle ablation model and a multi-needle combined ablation model.
The method according to claim 9, wherein the single-needle ablation model is displayed in the three-dimensional display window according to the single ablation needle inserted into the lesion based on the registration result and the ablation parameter. 3D model of the ablation lesion.
The method according to claim 9, wherein the multi-needle combined ablation model is based on the results of the registration and the ablation parameters, according to the position where the at least two ablation needles inserted into the lesion sequentially or simultaneously The three-dimensional model of the ablation lesion displayed in the three-dimensional display window.
The method according to claim 1, wherein the target viewing angle of the ablation region that meets the predetermined requirements for ablation of the lesion comprises:

The viewing angle at which the ratio of the projected area of the ablation area to the projected area of the three-dimensional model of the lesion is the smallest, or the viewing angle at which the ratio of the projected area of the ablation area to the projected area of the three-dimensional model of the lesion is lower than a predetermined threshold.
The method according to claim 1, wherein the target viewing angle of the ablation region that meets the predetermined requirements for ablation of the lesion comprises:

The viewing angle at which the ratio of the projected area of the ablation region to the projected area of the three-dimensional model of the ablation lesion is the smallest, or the ratio of the projected area of the ablation region to the projected area of the three-dimensional model of the ablation lesion is lower than a predetermined threshold perspective.
The method according to claim 1, wherein when multiple viewing angles meet the predetermined requirement, the target viewing angle is the viewing angle with the smallest required rotation angle.
The method according to claim 1, wherein, in the three-dimensional display window, the three-dimensional model of the lesion and the current view of the three-dimensional model of the ablation focus are automatically rotated from the current viewing angle to the ablation area that satisfies the ablation area. The target view of the lesion's predetermined requirements, including:

determining the first coordinate of the projection center position of the unablated region of the three-dimensional model of the lesion under the current viewing angle;

determining the second coordinate of the projection center position of the unablated area of the three-dimensional model of the lesion under the target viewing angle;

Determine the included angle between the line connecting the first coordinate and the coordinate origin and the line connecting the second coordinate and the coordinate origin, and use the included angle as the rotation angle of the rotation.
The method according to claim 1, wherein, in the three-dimensional display window, the three-dimensional model of the lesion and the current view of the three-dimensional model of the ablation focus are automatically rotated from the current viewing angle to the ablation area that satisfies the ablation area. The target view of the lesion's predetermined requirements, including:

determining the third coordinate of the projection center position of the three-dimensional model of the lesion under the current viewing angle;

determining the second coordinate of the projection center position of the unablated area of the three-dimensional model of the lesion under the target viewing angle;

Determine the included angle between the line connecting the third coordinate and the coordinate origin and the line connecting the second coordinate and the coordinate origin, and use the included angle as the rotation angle of the rotation.
The method of claim 1, wherein the registering comprises:

aligning the ultrasound image with corresponding slices in the three-dimensional image;

A coordinate transformation matrix from the ultrasound image space to the three-dimensional image space is determined according to the coordinates of the points in the ultrasound image and the coordinates of the corresponding points in the three-dimensional image.
The method of claim 17, wherein the aligning comprises at least one of the following:

Receive user's manual alignment;

identifying the same tissue in the ultrasound image and the three-dimensional image for automatic alignment; or

The imaged positions of the in vitro markers in the three-dimensional image are identified, and the positions of the in vitro markers in the space of the ultrasound image are determined based on the magnetic navigation for automatic alignment.
The method according to claim 1, wherein when the three-dimensional image is a three-dimensional ultrasound image reconstructed based on magnetic navigation position information, the registration comprises: based on the ultrasound image and the three-dimensional ultrasound image location information for automatic registration.
The method according to claim 1, wherein the acquired ablation parameters include parameters reflecting the shape and position of the ablation focus.
The method according to claim 20, wherein the parameters reflecting the shape and position of the ablation focus include at least one of the following:

The long and short diameters of the ellipsoid ablation lesions, the location of the heat source, the depth of the ablation path, and the angle of the puncture frame.
The method according to claim 1, wherein the displaying the three-dimensional model of the ablation lesion in the three-dimensional display window according to the registration result and the acquired ablation parameters comprises:

determining the coordinates of the center of the ablation focus in the ultrasound image, and determining the coordinates of the center of the ablation focus in the three-dimensional image according to the registration matrix from the ultrasound image to the three-dimensional image;

According to the coordinates of the center of the ablation focus in the three-dimensional image and the size of the ablation focus, a three-dimensional model of the ablation focus is drawn in the three-dimensional image space.
The method according to claim 22, wherein the determining the coordinates of the center of the ablation focus in the ultrasound image comprises:

The coordinates of the center of the ablation focus in the ultrasound image are determined according to the angle of the puncture frame and the depth of the ablation path.
The method according to claim 22, wherein the determining the coordinates of the center of the ablation focus in the ultrasound image comprises:

The coordinates of the center of the ablation focus in the ultrasound image are determined according to the positioning sensor disposed on the ablation needle.
The method according to claim 1, wherein the shape of the three-dimensional model of the ablation focus comprises ellipsoid, spherical or cylindrical.
The method according to claim 1, wherein the method further comprises:

Two or more three-dimensional display windows are displayed on the display interface, wherein one three-dimensional display window displays the current view, and the other three-dimensional display windows display views from other perspectives.
The method according to claim 26, wherein the two or more three-dimensional display windows include a main window and an auxiliary window, the main window is used to display the current view, and the auxiliary window is used to display the current view. The view from the other viewing angle is displayed, and the size of the main window is larger than the size of the auxiliary window.
The method according to claim 27, wherein the auxiliary window is superimposed and displayed at a corner position of the main window.
27. The method of claim 26, wherein the other viewing angle comprises a reverse viewing angle of the current viewing angle.
The method according to claim 26, wherein when the rotation of the current view occurs, the view in the other viewing angle is fixed.
The method according to claim 26, wherein when the rotation of the current view occurs, the views under the other viewing angles are rotated synchronously accordingly.
The method according to claim 1, wherein the displaying the three-dimensional model of the lesion in the three-dimensional display window comprises:

segmenting the three-dimensional contour of the lesion in the three-dimensional image;

performing surface reconstruction according to the three-dimensional contour obtained by the segmentation to generate a three-dimensional model of the lesion;

According to the coordinates of the three-dimensional contour of the lesion in the three-dimensional image, the three-dimensional model of the lesion is displayed in the three-dimensional display window.
A three-dimensional display method, characterized in that the method comprises:

Display two or more three-dimensional display windows on the display interface;

displaying the current view in one of the two or more three-dimensional display windows, wherein the current view includes the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion under the current viewing angle;

Views from other perspectives are displayed in the remaining three-dimensional display windows of the two or more three-dimensional display windows, and the views from other perspectives include the three-dimensional model of the lesion and the three-dimensional image of the ablation lesion from other perspectives Model.
The method according to claim 33, wherein the current view and/or the views under the other viewing angles are further displayed with ultrasound images.
The method of claim 33, wherein the method further comprises:

An ultrasound window and a fusion display window are displayed on the display interface, the ultrasound window is used for displaying an ultrasound image, and the fusion display window is used for displaying a fusion image of the ultrasound image and a three-dimensional image including the lesion.
An ultrasound imaging system, comprising:

Ultrasound probe;

a transmitting circuit, used to excite the ultrasonic probe to transmit ultrasonic waves to the lesion;

a receiving circuit, configured to control the ultrasonic probe to receive the ultrasonic echo returned from the lesion to obtain an ultrasonic echo signal;

processor for:

processing the ultrasonic echo signal to obtain an ultrasonic image;

registering the ultrasound image with a pre-acquired three-dimensional image containing the lesion;

Segmenting the lesion in the three-dimensional image containing the lesion, and displaying the three-dimensional model of the lesion in the three-dimensional display window of the display based on the result of the registration and the result of the segmentation;

acquiring ablation parameters of the ablation lesion, and displaying the three-dimensional model of the ablation lesion in the three-dimensional display window according to the registration result and the acquired ablation parameters;

Determine the ablation area according to the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion;

When the rotation condition is met, the current view of the three-dimensional display window is rotated from the current viewing angle to a target viewing angle in which the ablation area meets the predetermined requirements for ablating the lesion, and the three-dimensional model of the lesion and all the lesions in the target viewing angle are displayed. The three-dimensional model of the ablation lesion;

The display is used for displaying the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion.
The ultrasound imaging system according to claim 36, wherein the ablation area is an overlapping area of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion.
The ultrasound imaging system according to claim 36 or 37, wherein the satisfying the rotation condition comprises: the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfies the rotation condition, or receiving a user Enter the rotation command.
The ultrasound imaging system according to claim 38, wherein the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion satisfying a rotation condition comprises:

In the three-dimensional display window, the inclusion of the spatial position of the three-dimensional model of the lesion and the spatial position of the three-dimensional model of the ablation lesion in any viewing angle satisfies a preset condition.
The ultrasound imaging system according to claim 38, wherein the spatial relationship between the three-dimensional model of the lesion and the three-dimensional model of the ablation focus satisfies a rotation condition, comprising:

Under the current viewing angle, the projected area of the ablation area in the three-dimensional display window and the projected area of the three-dimensional model of the lesion, or the projected area of the ablation area and the projected area of the ablation lesion meet the predetermined requirements Set conditions.
The ultrasonic imaging system according to claim 40, wherein the meeting a preset condition comprises:

The proportion of the projected area of the ablation region to the projected area of the three-dimensional model of the lesion exceeds a predetermined threshold, or

The proportion of the projected area of the ablation region to the projected area of the three-dimensional model of the ablation lesion exceeds a predetermined threshold.
The ultrasound imaging system according to claim 36, wherein the target angle of view for the ablation region to meet the predetermined requirements for ablation of the lesion comprises:

The viewing angle at which the ratio of the projected area of the ablation area to the projected area of the three-dimensional model of the lesion is the smallest, or the viewing angle at which the ratio of the projected area of the ablation area to the projected area of the three-dimensional model of the lesion is lower than a predetermined threshold;

Or, the viewing angle at which the ratio of the projected area of the ablation region to the projected area of the three-dimensional model of the ablation lesion is the smallest, or the ratio of the projected area of the ablation region to the projected area of the three-dimensional model of the ablation lesion is lower than a predetermined value Threshold viewing angle.
The ultrasound imaging system according to claim 36, wherein the processor automatically rotates the current view of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion in the three-dimensional display window from the current viewing angle to The target viewing angle of the ablation area that meets the predetermined requirements for ablation of the lesion, including:

determining the first coordinate of the projection center position of the unablated region of the three-dimensional model of the lesion under the current viewing angle;

determining the second coordinate of the projection center position of the unablated area of the three-dimensional model of the lesion under the target viewing angle;

Determine the included angle between the line connecting the first coordinate and the coordinate origin and the line connecting the second coordinate and the coordinate origin, and use the included angle as the rotation angle of the rotation.
The ultrasound imaging system according to claim 36, wherein the processor automatically rotates the current view of the three-dimensional model of the lesion and the three-dimensional model of the ablation lesion in the three-dimensional display window from the current viewing angle to The target viewing angle of the ablation area that meets the predetermined requirements for ablation of the lesion, including:

determining the third coordinate of the projection center position of the three-dimensional model of the lesion under the current viewing angle;

determining the second coordinate of the projection center position of the unablated area of the three-dimensional model of the lesion under the target viewing angle;

Determine the included angle between the line connecting the third coordinate and the coordinate origin and the line connecting the second coordinate and the coordinate origin, and use the included angle as the rotation angle of the rotation.
The ultrasound imaging system of claim 36, wherein the processor is further configured to:

Two or more three-dimensional display windows are displayed on the display interface of the display, wherein one three-dimensional display window displays the current view, and the other three-dimensional display windows display views from other perspectives.
The ultrasound imaging system according to claim 45, wherein the two or more three-dimensional display windows include a main window and an auxiliary window, the main window is used to display the current view, and the auxiliary window For displaying the view from the other viewing angle, the size of the main window is larger than the size of the auxiliary window.
The ultrasound imaging system according to claim 45, wherein when the current view rotates, the views under the other viewing angles are fixed, or the views under the other viewing angles are rotated synchronously therewith .