GB2616029A

GB2616029A - Improvements in or relating to the navigation of an ultrasound probe device

Info

Publication number: GB2616029A
Application number: GB2202531.6A
Authority: GB
Inventors: Schewel Jury; Schewel Dimitry
Original assignee: Robe GmbH
Current assignee: Robe GmbH
Priority date: 2022-02-24
Filing date: 2022-02-24
Publication date: 2023-08-30
Also published as: GB202202531D0

Abstract

A computer-implemented method of defining a coordinate system comprises: receiving a sequence of ultrasound images from an ultrasound probe device, each ultrasound image representing a respective target view 804; receiving, from a controller of the ultrasound probe device, at least one actuator datum associated with each ultrasound image 806; and defining a coordinate system for the sequence of ultrasound images based on the at least one actuator datum associated with each ultrasound image, wherein the at least one actuator datum associated with a designated ultrasound image of the sequence of ultrasound images defines the origin of the coordinate system 808. There is also a computer-implemented method of calculating actuator operation instructions comprising defining a coordinate system. In one embodiment, instructions for moving the ultrasound probe device from a placement corresponding to a designated ultrasound image to any one of the remaining ultrasound images is calculated from the coordinate system. In another embodiment, movement of the ultrasound probe device through a plurality of placements, each placement associated with a respective target view, is calculated from the coordinate system.

Description

IMPROVEMENTS IN OR RELATING TO THE NAVIGATION OF AN ULTRASOUND PROBE DEVICE

Technical Field

The present disclosure relates to a computer-implemented method of defining a coordinate system and computer-implemented methods of calculating actuator operation instruction for moving an ultrasound probe device. Associated methods and apparatus, and corresponding computer programs, are also disclosed.

Background

Echocardiography, more generally ultrasound image data, can be used in diagnostics, e.g., to provide images of intracardiac structures such as heart valves, left or right atria and ventricle, or the interatrial and interventricular septum.

Additionally, echocardiography can be used to assist in the performance of interventional or operative procedures, such as the treatment of structural heart disease or arrhythmias, both by providing adequate visualization of the respective intracardiac structures during pre-procedure planning (preplanning) and the procedure itself, and by allowing an interventional cardiologist or a surgeon to accurately navigate various catheters, guide wires and other devices within a heart for optimal operative results.

Typically, this requires an echocardiographer to operate a flexible echocardiographic instrument, e.g., a transesophageal echocardiography (TEE) probe or an intracardiac echocardiography (ICE) catheter, with the same degree of accuracy and precision during preplanning and a procedure, in order to return the flexible echocardiographic instrument to the same site(s) of interest. This requires a high degree of skill and ordinarily can be a time-consuming challenge which leads to longer procedures. Additionally, procedures are becoming more complex which requires more time still to avoid reduced image quality which could otherwise lead to sub-optimal results.

Such lengthening of procedures is undesirable because it results in a dramatic increase in exposure to radioactive fluoroscopy by both the surgeon, and particularly the echocardiographer, and because it risks procedure reproducibility.

The listing or discussion of a prior-published document or any background in this specification should not necessarily be taken as an acknowledgement that the document or background is part of the state of the art or is common general knowledge. One or more aspects/examples of the present disclosure may or may not address one or more of the background issues.

Summary

According to a first aspect, there is provided a computer-implemented method of defining a coordinate system, comprising: receiving a sequence of ultrasound images from an ultrasound probe device, each ultrasound image representing a respective target view; receiving, from a controller of the ultrasound probe device, at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images; and defining a coordinate system for the sequence of ultrasound images based on the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images, wherein the at least one actuator datum associated with a designated ultrasound image of the sequence of ultrasound images defines the origin of the coordinate system.

According to a second aspect, there is provided a computer-implemented method of calculating actuator operation instructions for moving an ultrasound probe device, comprising: defining a coordinate system; and calculating, from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device from a placement corresponding to a designated ultrasound image of a sequence of ultrasound images to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.

According to a third aspect, there is provided a computer-implemented method of calculating actuator operation instructions for moving an ultrasound probe device, comprising: defining a coordinate system; and calculating, from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device through a plurality of placements, each placement associated with a respective target view.

The computer-implemented method of the second or the third aspect may comprise the computer-implemented method of the first aspect to define the coordinate system.

Defining the coordinate system may comprise: converting the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images into a set of respective coordinates; determining a null position from the at least one actuator datum associated with the designated ultrasound image of the sequence of ultrasound images; and referencing the actuator data associated with each remaining ultrasound image of the sequence of ultrasound images to the null position.

The at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images may comprise data for a path of travel linking each target view. The actuator operation instructions may be stored to a memory.

The computer-implemented method of the first aspect may further comprise: receiving at least one further ultrasound image from the ultrasound probe device, the at least one further ultrasound image representing a further target view; receiving, from the controller of the ultrasound probe device, at least one actuator datum for the least one further ultrasound image; and updating the coordinate system to include coordinates for the at least one further ultrasound image.

The computer-implemented method of the second aspect may further comprise: establishing that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images; receiving an instruction to move the ultrasound probe device to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing the actuator operation instructions to the controller to move the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.

Establishing that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images may be based on a comparison of ultrasound image data.

The computer-implemented method of the second aspect may further comprise: establishing that the controller has moved the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing an instruction to the ultrasound probe device to capture at least one ultrasound image.

The placement associated with any one of the remaining ultrasound images of the sequence of ultrasound image may be: a placement corresponding to the previously received at least one actuator datum for any one of the remaining ultrasound images of the sequence of ultrasound image, or a placement in a predetermined region of the coordinate system, for acquiring a further instance of any one of the remaining ultrasound images of the sequence of ultrasound image.

The predetermined region of the coordinate system may correspond to a region suitable for imaging an intracardiac or an extracardiac structure, preferably a region suitable for imaging the right atrium, the right ventricle, the left atrium, or the left ventricle, or a region suitable for bicaval imaging.

The controller may comprise at least one actuator configured to move the ultrasound probe device in at least one degree of freedom.

The controller may comprise at least two actuators configured to move the ultrasound probe device in at least two degrees of freedom, optionally five actuators for five degrees of freedom, and wherein the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images comprises a position and an orientation.

The actuator operation instructions may comprise instructions for linear and/or angular movement of the ultrasound probe device.

The computer-implemented method of the second aspect may further comprise receiving an indication to permit manually guided movement of the ultrasound probe device.

The computer-implemented method of the second aspect may further comprise: calculating, from the coordinate system, actuator operation instructions for the controller to return the ultrasound probe device from the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images.

The computer-implemented method of the second aspect may further comprise: updating the coordinate system to include coordinates for at least one further ultrasound image; and calculating, from the coordinate system, actuator operation instructions for the controller to move the ultrasound probe device among the placements associated with each ultrasound image of the sequence of ultrasound images and the at least one further ultrasound image.

The designated ultrasound image of the sequence of ultrasound images may be the first ultrasound image in the sequence of ultrasound images.

The computer-implemented method of the third aspect may further comprise: establishing that the controller has moved the ultrasound probe device to a placement corresponding to a designated ultrasound image; providing the actuator operation instructions to the controller to move the ultrasound probe device through the plurality of placements; and for each placement of the plurality of placements, providing an instruction to the ultrasound probe device to capture ultrasound image data.

The computer-implemented method of the third aspect may further comprise: providing instructions for the controller to move the ultrasound probe device through the plurality of placements a predetermined number of times, and/or providing instructions to the ultrasound probe device to capture multiple ultrasound images at each placement of the plurality of placements.

The computer-implemented method of the third aspect may further comprise: receiving captured ultrasound image data from the ultrasound probe device; and constructing a 3D model from the received ultrasound image data.

The computer-implemented method of the third aspect may further comprise: receiving an indication of one or more target views within the 3D model; and calculating, from the coordinate system, actuator operation instructions for the controller to move the ultrasound probe device from the placement corresponding to the designated ultrasound image to a placement associated with any one of the one or more target views within the 3D model.

The one or more target views within the 3D model may comprise a view of an intracardiac or extracardiac structure, preferably one or more of the right atrium, the right ventricle, the left atrium, the left ventricle, the bicaval view, the mitral valve, the tricuspid valve, and the left atrial appendage.

The computer-implemented method of the first, second or third aspect may further comprise: referencing the placement of at least one further probe device to the coordinate system; and calculating, from the coordinate system, actuator operation instructions for a controller to move the at least one further probe device in coordination with the ultrasound probe device.

The computer-implemented method of the first, second or third aspect may further comprise: establishing that the controller for the at least one further probe device has located the at least one further probe device at a placement within the coordinate system; receiving an instruction to move the at least one further probe device to a placement associated with a placement of the ultrasound probe device; and providing the actuator operation instructions to the controller for the at least one further probe device to move the at least one further probe device in coordination with the ultrasound probe device.

The at least one further probe device may be a fluoroscopy device, optionally a C-Arm fluoroscope, or an X-ray machine, or a echocardiography transducer, optionally a transcutaneous echocardiography transducer, an intercardiac ECG, ICE, device, an MRI scanner, or a CT scanner.

The ultrasound probe device may an echocardiography probe, optionally a transesophageal or transthoracic echocardiography probe.

According to a fourth aspect there is provided an apparatus comprising at least one processor and at least one memory storing computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least perform the computer-implemented method of the first, second or third aspect.

According to a fifth aspect there is provided a controller for an ultrasound probe device, the controller comprising the apparatus of the fourth aspect.

According to a sixth aspect there is provided a system comprising the controller of the fifth aspect and an ultrasound probe device.

According to a seventh aspect there is provided a method of defining a coordinate system for a patient, comprising the steps of: inserting an ultrasound probe device into the patient; acquiring a sequence of ultrasound images from the ultrasound probe device, each ultrasound image representing a respective target view; receiving, from a controller of the ultrasound probe device, at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images; and defining, using an apparatus comprising at least one processor and at least one memory storing computer program code, a coordinate system for the sequence of ultrasound images based on the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images, wherein the at least one actuator datum associated with a designated ultrasound image of the sequence of ultrasound images defines the origin of the coordinate system.

According to an eight aspect there is provided a method of moving an ultrasound probe device within a patient, comprising: defining a coordinate system; calculating, using an apparatus comprising at least one processor and at least one memory storing computer program code and from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device from a placement corresponding to a designated ultrasound image of a sequence of ultrasound images to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images, each ultrasound image representing a respective target view; inserting the ultrasound probe device into the patient; establishing that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images; designating a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing an instruction to the apparatus to provide the actuator operation instructions to the controller to move the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.

According to an ninth aspect there is provided a method of moving an ultrasound probe device within a patient, comprising: defining a coordinate system; calculating, using an apparatus comprising at least one processor and at least one memory storing computer program code and from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device through a plurality of placements, each placement associated with a respective target view; inserting the ultrasound probe device into the patient; establishing that the controller has returned the ultrasound probe device to a placement corresponding to a designated ultrasound image, the designated ultrasound image representing a designated target view; and providing an instruction to the apparatus to provide the actuator operation instructions from the memory to the controller to: move the ultrasound probe device through the plurality of positions; and, for each position of the plurality of positions, providing an instruction to the ultrasound probe device to capture ultrasound image data.

The optional features described in relation to each aspect are applicable to, and may be combined with, each other aspect were compatible.

The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated or understood by the skilled person.

Corresponding computer programs for implementing one or more steps of the methods disclosed herein are also within the present disclosure and are encompassed by one or more of the described examples.

One or more of the computer programs may, when run on a computer, cause the computer to configure any apparatus, including a battery, circuit, controller, or device disclosed herein or perform any method disclosed herein. One or more of the computer programs may be software implementations, and the computer may be considered as any appropriate hardware, including a digital signal processor, a microcontroller, and an implementation in read only memory (ROM), erasable programmable read only memory (EPROM) or electronically erasable programmable read only memory (EEPROM), as non-limiting examples. The software may be an assembly program.

One or more of the computer programs may be provided on a computer readable medium, which may be a physical computer readable medium such as a disc or a memory device, or may be embodied as a transient signal. Such a transient signal may be a network download, including an internet download.

The present disclosure includes one or more corresponding aspects, examples or features in isolation or in various combinations whether or not specifically stated (including claimed) in that combination or in isolation. Corresponding means for performing one or more of the discussed functions are also within the present disclosure.

Throughout the present specification, descriptors relating to movement such as "forwards", "backwards" or "reverse", "linear", "translate", and "rotate", descriptors relating to position such as "location" and "point", and descriptors relating to orientation such as "direction", "pose" and "rotation", as well as any adjective and adverb derivatives thereof, are used in the sense of the movement, position and orientation discussed in the context of the examples disclosed herein. However, such descriptors are not intended to be in any way limiting to an intended use of the described or claimed invention.

The above summary is intended to be merely exemplary and non-limiting. Brief Description of the Figures A description is now given, by way of example only, with reference to the accompanying schematic drawings, in which: Figure 1 shows a flow diagram of a method of collecting ultrasound image data from one or more target views.

Figure 2 shows a flow diagram of a computer-implemented method of defining a coordinate system.

Figure 3 shows a flow diagram of a method of returning to one or more target views.

Figure 4 shows a flow diagram of a computer-implemented method of calculating actuator operation instructions for moving an ultrasound probe device.

Figure 5 shows a flow diagram of a method of collecting ultrasound image data.

Figure 6 shows a flow diagram of another computer-implemented method of calculating actuator operation instructions for moving an ultrasound probe device.

Figure 7 shows in schematic form an example apparatus.

Figure 8 shows a flow diagram of a method of defining a coordinate system for a patient.

Figure 9 shows a flow diagram of a method moving an ultrasound probe device within a patient.

Figure 10 shows a flow diagram of another method of moving an ultrasound probe device within a patient.

Detailed Description

Figure 1 shows a flow diagram of a method 100 of collecting ultrasound image data from one or more target views, which may be implemented as part of planning an interventional or operative procedure. The method 100 comprises: installing 102 an echocardiography probe and associated controller; manipulating 104 the degrees of freedom afforded by the controller to the echocardiography probe; navigating 106, by the controller, the echocardiography probe to a reference position (for example, the four-chamber view); defining and storing 108 an image captured at the reference position; navigating 110, by the controller, the echocardiography probe to a target view; saving 112 the coordinates from the controller for the target view; repeating 114 steps 110-112 as often as necessary; and using 116 the saved views/positions as required.

Preplanning for an interventional or operative procedure typically relies on manual guidance of an ultrasound probe device (e.g., an echocardiography probe such as a transesophageal or transthoracic echocardiography probe). The time required to manually return the ultrasound prove device to a previously visited location during, for example, the operative procedure can be time-consuming in order to maximise the precision of the return. Consequently, practitioners within the operating theatre may risk unsafe levels of radiation exposure.

For automated navigation a reference point must first be defined. This reference point is typically user-defined and corresponds to a point for a designated view (e.g., for the four-chamber view) navigated to during preplanning. Relative to the reference point, which may also be termed a zero point or origin, a coordinate system is established. Activation of any motor used by the controller to move the echocardiography probe leads to a change of the position of the echocardiography probe relative to the reference point that can be exactly defined within the coordinate system. Saving multiple positions within the coordinate system is achieved by saving the associated controller actuator data (e.g., data defining the positions of one or more motors for the echocardiography probe). By these means, the controller can automatically navigate the echocardiography probe to a predefined position or positions within the coordinate system.

Figure 2 shows a flow diagram of a computer-implemented method 220 of defining a coordinate system. The method 200 comprises: receiving 222 a sequence of ultrasound images from an ultrasound probe device, each ultrasound image representing a respective target view; receiving 224, from a controller of the ultrasound probe device, at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images; and defining 226 a coordinate system for the sequence of ultrasound images based on the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images, wherein the at least one actuator datum associated with a designated ultrasound image of the sequence of ultrasound images defines the origin of the coordinate system.

Defining a coordinate system as described above supports the automated navigation of an ultrasound probe device and therefore improves the effectiveness of preplanning exercises. Associated advantages may include one or more of enhanced reproducibility of ultrasound probe device navigation, improved accuracy and precision of ultrasound probe device scan (sweep) routines, a reduction in the time required to complete a procedure, and reduced radiation exposure to procedure practitioners.

The at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images comprises data for a path of travel linking each target view: for example, the path of travel taken by an echocardiography probe within a patient during pre-procedure planning. In this way, automatic navigation of the echocardiography probe on the basis of the coordinate system may follow the manual navigation path to minimise patient risk during the subsequent interventional or operative procedure.

The method 200 may further comprise: receiving at least one further ultrasound image from the ultrasound probe device, the at least one further ultrasound image representing a further target view; receiving, from the controller of the ultrasound probe device, at least one actuator datum for the least one further ultrasound image; and updating the coordinate system to include coordinates for the at least one further ultrasound image. In this way, the coordinate system may be updated to account for one or more supplementary preplanning exercises, for example.

Figure 3 shows a flow diagram of a method 330 of returning to one or more target views that applies the coordinate system concept. The method comprises: installing 332 an echocardiography probe and associated controller; setting 334 up the reference position; using 336 the controller to navigate the echocardiography probe semi-automatically to a standardised view; optimizing 338 the standardised view by manually adjusting each degree of freedom of the echocardiography probe afforded by the controller; saving 340 the coordinates from the controller for the standardised view; navigating 342, by the controller, the echocardiography probe to the next standardised view; repeating 344 steps 340-342 as often as necessary; and using 346 the saved views/positions as required.

In an example, every standardized view (e.g., for the two-, three-, or four-chamber view, the bicaval view, etc.) has predetermined coordinates for each degree of freedom afforded by the controller to the echocardiography probe. These coordinates are set in relation to the origin of the coordinate system.

Returning the echocardiography probe to one or more target views as described above exemplifies a computer-implemented method 450 of calculating actuator operation instructions for moving an ultrasound probe device. As shown in Figure 4, the method 405 comprises: defining 452 a coordinate system; and calculating 454, from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device from a placement corresponding to a designated ultrasound image of a sequence of ultrasound images to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.

The method 450 may comprise the method shown in Figure 2 and any steps associated with that method to define the coordinate system. For example, the actuator operation instructions may replicate or smooth a path of travel represented by the actuator data for the sequence of ultrasound images.

The actuator operation instructions may be stored to a memory for later use. The designated ultrasound image of the sequence of ultrasound images may designated based on a user-defined protocol, for example, the first ultrasound image in the sequence of ultrasound images.

Forward and inverse kinematics techniques known in the art may be applied to arrive at the coordinate system and the actuator operation instructions. For example, the joint parameters corresponding to the available degrees of freedom of the ultrasound probe device may first be determined to provide a desired configuration (e.g., position and rotation). Using forward kinematics techniques, the joint parameters are then used to compute the configuration of the kinematic chain (i.e., the assembly comprising the ultrasound probe device, its actuators, and related kinematic pairs). Using inverse kinematics techniques, the computation is reversed to determine the joint parameters that achieve a desired configuration. This enables transformation of a user-defined motion plan (e.g., prepared by a cardiologist and echocardiographer) into joint actuator trajectories and velocities for moving the ultrasound probe device to locations suitable for imaging one or more target views.

The method 450 may further comprise: establishing that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images; receiving an instruction to move the ultrasound probe device to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing the actuator operation instructions to the controller to move the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.

Establishing that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images may be based on a comparison of ultrasound image data. Image recognition or object detection techniques may be utilised in this respect, which may be neural network-based or non-neural network-based.

The method 450 may further comprise establishing that the controller has moved the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing an instruction to the ultrasound probe device to capture at least one ultrasound image.

The placement associated with an ultrasound image of the sequence of ultrasound images may not necessarily match the placement of the ultrasound probe device when the ultrasound image was first captured. That is, a user may plan for the controller to move the ultrasound probe device close to the initial placement (i.e., close to the placement corresponding to the target view) to provide a complementary view of a particular target. Interpolation and/or other estimation techniques may be used to construct kinematic data missing from the coordinate system to provide this functionality.

Accordingly, the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound image is: a placement corresponding to the previously received at least one actuator datum for any one of the remaining ultrasound images of the sequence of ultrasound image, or a placement in a predetermined region of the coordinate system, for acquiring a further instance of any one of the remaining ultrasound images of the sequence of ultrasound image. The predetermined region of the coordinate system may correspond to a region suitable for imaging an intracardiac or extracardiac structure, preferably a region suitable for imaging the right atrium, the right ventricle, the left atrium, or the left ventricle, or a region suitable for bicaval imaging.

While the coordinate system is defined based on at least one actuator datum associated with each ultrasound image of a sequence of ultrasound images (i.e., it has at least one dimension), it may be defined for up to all (i.e., up to two, three, four, five, or six) available degrees of freedom -position and/or orientation -of the ultrasound probe device. The coordinate system may comprise a 'global' or 'world' coordinate system (e.g., a Cartesian coordinate system for linear/translational movements) to which a local coordinate system (e.g., that defines the applicable Euler angles for direction/orientation/rotation) is referenced. Examples of coordinate systems and their respective transformations are known in the art and are encompassed by the methods disclosed herein.

Accordingly, the controller may comprise at least one actuator configured to move the ultrasound probe device in at least one degree of freedom. The controller may be configured to provide actuator data for each degree of freedom.

The controller may comprise at least two actuators configured to move the ultrasound probe device in at least two degrees of freedom, optionally five actuators for five degrees of freedom, and the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images may comprise a position and an orientation. Alternatively, the placement may comprise positional information only, depending on the configuration of the controller and its actuators.

Further defining the placement of the ultrasound probe device in this manner is beneficial to calculate more complex linear and rotational manoeuvres from the coordinate system; e.g., the actuator operation instructions may thereafter comprise instructions for translational and/or rotational movement of the ultrasound probe device to better conform to a manually-defined path of travel.

The method 450 may further comprise receiving an indication to permit manually guided movement of the ultrasound probe device. In this way, placement of the ultrasound probe device can be optimised by an echocardiographer (for example) to maintain patient safety and to maximise the quality of captured ultrasound images.

To allow for the automated return of the ultrasound probe device to the reference position, the method 450 may further comprise calculating, from the coordinate system, actuator operation instructions for the controller to return the ultrasound probe device from the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images. The method 450 may also comprise updating the coordinate system to include coordinates for at least one further ultrasound image; and calculating, from the coordinate system, actuator operation instructions for the controller to move the ultrasound probe device among the placements associated with each ultrasound image of the sequence of ultrasound images and the at least one further ultrasound image. Updating the coordinate system to include coordinates for at least one further ultrasound image may follow from supplementary preplanning (cf. the discussion of the method shown in Figure 2) to return to a new saved view, a procedure to resolve missing kinematic data, or similar.

Figure 5 shows a flow diagram of a method 560 of collecting ultrasound image data that extends the coordinate system concept. The method 560 comprises: installing 562 an echocardiography probe and associated controller; setting up 564 the reference position; executing 566 a full heart sweep; analyzing 568 a 3D model based on the full heart sweep; defining and saving 570 optimal echocardiography probe coordinates for targeted views; and using 572 the saved coordinates for an interventional procedure.

The full heart sweep adopts a roadmap with predefined coordinates for the controller and echocardiography probe to scan the full heart. As an alternative to a full heart sweep, a focused sweep (e.g., of the mitral valve, the tricuspid valve, the left atrial appendage, etc.) may be executed instead. Either one loop from the beginning until the end of the roadmap is recorded or at each coordinate a loop is recorded. For example, at each coordinate a multi-beat loop with 2-3 cycles is recorded. Reconstruction of recorded echocardiograms into a 3D model enables user analysis to identify (e.g., via multiplanar analysis) one or more targeted views suitable for an interventional procedure.

Collecting ultrasound image data as described above builds upon another computer-implemented method 680 of calculating actuator operation instructions for moving an ultrasound probe device. As shown in Figure 6, the method 680 comprises: defining 682 a coordinate system; and calculating 684, from the coordinate system, actuator operation instructions for a controller to move an ultrasound probe device through a plurality of placements, each placement associated with a respective target view.

The method 680 may comprise the method shown in Figure 2 and any steps associated with that method to define the coordinate system.

The method 680 may comprise establishing that the controller has moved the ultrasound probe device to a placement corresponding to a designated ultrasound image; providing the actuator operation instructions to the controller to move the ultrasound probe device through the plurality of placements; and for each placement of the plurality of placements, providing an instruction to the ultrasound probe device to capture ultrasound image data.

The method 680 may further comprise providing instructions for the controller to move the ultrasound probe device through the plurality of placements a predetermined number of times, and/or providing instructions to the ultrasound probe device to capture multiple ultrasound images at each placement of the plurality of placements.

The method 680 may further comprise receiving captured ultrasound image data from the ultrasound probe device; and constructing a 3D model from the received ultrasound image data.

Automated control of an ultrasound probe device through a scan routine advantageously provides greater precision and accuracy (and, therefore, an improvement in the quality of a model derived from the scan routine) compared to an alternative solution in which the ultrasound probe device is manually navigated. This is especially true for an ultrasound probe device with, e.g., five of degrees of freedom, where an order of magnitude improvement in precision for each degree of freedom (compared to manual manipulation) scales to provide a several orders of magnitude increase in the number of potential scanning positions.

The method 680 may further comprise receiving an indication of one or more target views within the 3D model; and calculating, from the coordinate system, actuator operation instructions for the controller to move the ultrasound probe device from the placement corresponding to the designated ultrasound image to a placement associated with any one of the one or more target views within the 3D model. The one or more target views within the 3D model may comprise a view of an intracardiac or extracardiac structure, preferably one or more of the right atrium, the right ventricle, the left atrium, the left ventricle, the bicaval view, the mitral valve, the tricuspid valve, and the left atrial appendage.

To further extend the coordinate system concept, any of the methods described above may comprise referencing the placement of at least one further probe device to the coordinate system; and calculating, from the coordinate system, actuator operation instructions for a controller to move the at least one further probe device in coordination with the ultrasound probe device. The methods described above may further comprise establishing that the controller for the at least one further probe device has located the at least one further probe device at a placement within the coordinate system; receiving an instruction to move the at least one further probe device to a placement associated with a placement of the ultrasound probe device; and providing the actuator operation instructions to the controller for the at least one further probe device to move the at least one further probe device in coordination with the ultrasound probe device. In this way, the operation of an instrument that complements the ultrasound probe device may also benefit from improved positioning accuracy and coordination with the ultrasound probe device.

For example, the at least one further probe device may be a fluoroscopy device, optionally a C-Arm fluoroscope, or an X-ray machine, or an echocardiography transducer, optionally a transcutaneous echocardiography transducer, an intracardiac echocardiography, ICE, catheter, device, an MRI scanner, or a CT scanner.

Figure 7 shows in schematic form an example apparatus 790. The apparatus 760 comprises at least one processor 792 and at least one memory 794 storing computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to perform any of the methods described herein.

The apparatus 790 may be comprised by a controller for an ultrasound probe device as described above. The controller comprising the apparatus 790 may be provided as a system in combination with the ultrasound probe device.

Any of the examples described above may be practised on a patient in accordance with one or more of the following methods.

Figure 8 shows a flow diagram of a method 800 of defining a coordinate system for a patient. The method comprises: inserting 802 an ultrasound probe device into the patient; acquiring 804 a sequence of ultrasound images from the ultrasound probe device, each ultrasound image representing a respective target view; receiving 806, from a controller of the ultrasound probe device, at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images; and defining 808, using an apparatus comprising at least one processor and at least one memory storing computer program code, a coordinate system for the sequence of ultrasound images based on the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images, wherein the at least one actuator datum associated with a designated ultrasound image of the sequence of ultrasound images defines the origin of the coordinate system.

Figure 9 shows a flow diagram of a method 900 moving an ultrasound probe device within a patient. The method comprises defining 902 a coordinate system; calculating 904, using an apparatus comprising at least one processor and at least one memory storing computer program code and from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device from a placement corresponding to a designated ultrasound image of a sequence of ultrasound images to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images, each ultrasound image representing a respective target view; inserting 906 the ultrasound probe device into the patient; establishing 908 that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images; designating 910 a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing 912 an instruction to the apparatus to provide the actuator operation instructions to the controller to move the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.

Figure 10 shows a flow diagram of a further method 1000 of moving an ultrasound probe device within a patient. The method comprises: defining 1002 a coordinate system; calculating 1004, using an apparatus comprising at least one processor and at least one memory storing computer program code and from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device through a plurality of placements, each placement associated with a respective target view; inserting 1006 the ultrasound probe device into the patient; establishing 1008 that the controller has returned the ultrasound probe device to a placement corresponding to a designated ultrasound image, the designated ultrasound image representing a designated target view; and providing 1010 an instruction to the apparatus to provide the actuator operation instructions from the memory to the controller to: move the ultrasound probe device through the plurality of positions; and, for each position of the plurality of positions, providing 1012 an instruction to the ultrasound probe device to capture ultrasound image data.

The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole, in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that the disclosed aspects/embodiments may consist of any such individual feature or combination of features. In view of the foregoing description, it will be evident to a person skilled in the art that various modifications may be made within the scope of the disclosure.

Claims

Claims 1. A computer-implemented method of defining a coordinate system, comprising: receiving a sequence of ultrasound images from an ultrasound probe device, each ultrasound image representing a respective target view; receiving, from a controller of the ultrasound probe device, at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images; and defining a coordinate system for the sequence of ultrasound images based on the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images, wherein the at least one actuator datum associated with a designated ultrasound image of the sequence of ultrasound images defines the origin of the coordinate system.
2. A computer-implemented method of calculating actuator operation instructions for moving an ultrasound probe device, comprising: defining a coordinate system; and calculating, from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device from a placement corresponding to a designated ultrasound image of a sequence of ultrasound images to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.
3. A computer-implemented method of calculating actuator operation instructions for moving an ultrasound probe device, comprising: defining a coordinate system; and calculating, from the coordinate system, actuator operation instructions for a controller to move the ultrasound probe device through a plurality of placements, each placement associated with a respective target view.
4. The computer-implemented method of claim 1, wherein defining the coordinate system comprises: converting the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images into a set of respective coordinates; determining a null position from the at least one actuator datum associated with the designated ultrasound image of the sequence of ultrasound images; and referencing the actuator data associated with each remaining ultrasound image of the sequence of ultrasound images to the null position.
5. The computer-implemented method of claim 1 or claim 4, wherein the at least one actuator datum associated with each ultrasound image of the sequence of ultrasound images comprises data for a path of travel linking each target view.
6. The computer-implemented method of any of claims 1, 4 and 5, further comprising: receiving at least one further ultrasound image from the ultrasound probe device, the at least one further ultrasound image representing a further target view; receiving, from the controller of the ultrasound probe device, at least one actuator datum for the least one further ultrasound image; and updating the coordinate system to include coordinates for the at least one further ultrasound image.
7. The computer-implemented method of claim 2, further comprising: establishing that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images; receiving an instruction to move the ultrasound probe device to a placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing the actuator operation instructions to the controller to move the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images.
8. The computer-implemented method of claim 7, wherein establishing that the controller has returned the ultrasound probe device to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images is based on a comparison of ultrasound image data.
9. The computer-implemented method of claim 7 or claim 8, further comprising: establishing that the controller has moved the ultrasound probe device to the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images; and providing an instruction to the ultrasound probe device to capture at least one ultrasound image.
10. The computer-implemented method of any of claims 7-9, wherein the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound image is: a placement corresponding to the previously received at least one actuator datum for any one of the remaining ultrasound images of the sequence of ultrasound image, or a placement in a predetermined region of the coordinate system, for acquiring a further instance of any one of the remaining ultrasound images of the sequence of ultrasound image.
11. The computer-implemented method of any of claims 7-10, wherein the controller comprises at least one actuator configured to move the ultrasound probe device in at least one degree of freedom.
12. The computer-implemented method of claim 11, wherein the controller comprises at least two actuators configured to move the ultrasound probe device in at least two degrees of freedom, optionally five actuators for five degrees of freedom, and wherein the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images comprises a position and an orientation.
13. The computer-implemented method of any of claims 7-12, further comprising receiving an indication to permit manually guided movement of the ultrasound probe device.
14. The computer-implemented method of any of claims 2 and 7-13, further comprising: calculating, from the coordinate system, actuator operation instructions for the controller to return the ultrasound probe device from the placement associated with any one of the remaining ultrasound images of the sequence of ultrasound images to the placement corresponding to the designated ultrasound image of the sequence of ultrasound images.
15. The computer-implemented method of any of claims 2 and 7-13, further comprising: updating the coordinate system to include coordinates for at least one further ultrasound image; and calculating, from the coordinate system, actuator operation instructions for the controller to move the ultrasound probe device among the placements associated with each ultrasound image of the sequence of ultrasound images and the at least one further ultrasound image.
16. The computer-implemented method of any of claims 2 and 7-15, wherein the designated ultrasound image of the sequence of ultrasound images is the first ultrasound image in the sequence of ultrasound images.
17. The computer-implemented method of claim 3, further comprising: establishing that the controller has moved the ultrasound probe device to a placement corresponding to a designated ultrasound image; providing the actuator operation instructions to the controller to move the ultrasound probe device through the plurality of placements; and for each placement of the plurality of placements, providing an instruction to the ultrasound probe device to capture ultrasound image data.
18. The computer-implemented method of claim 17, further comprising: providing instructions for the controller to move the ultrasound probe device through the plurality of placements a predetermined number of times, and/or providing instructions to the ultrasound probe device to capture multiple ultrasound images at each placement of the plurality of placements.
19. The computer-implemented method of claim 17 or claim 18, further comprising: receiving captured ultrasound image data from the ultrasound probe device; and constructing a 3D model from the received ultrasound image data.
20. The computer-implemented method of claim 19, further comprising: receiving an indication of one or more target views within the 3D model; and calculating, from the coordinate system, actuator operation instructions for the controller to move the ultrasound probe device from the placement corresponding to the designated ultrasound image to a placement associated with any one of the one or more target views within the 3D model.
21. The computer-implemented method of any preceding claim, further comprising: referencing the placement of at least one further probe device to the coordinate system; and calculating, from the coordinate system, actuator operation instructions for a controller to move the at least one further probe device in coordination with the ultrasound probe device.
22. The computer-implemented method of claim 21, further comprising: establishing that the controller for the at least one further probe device has located the at least one further probe device at a placement within the coordinate system; receiving an instruction to move the at least one further probe device to a placement associated with a placement of the ultrasound probe device; and providing the actuator operation instructions to the controller for the at least one further probe device to move the at least one further probe device in coordination with the ultrasound probe device.
23. The computer-implemented method of any preceding claim, wherein the ultrasound probe device is an echocardiography probe, optionally a transesophageal or transthoracic echocardiography probe.
24. An apparatus comprising at least one processor and at least one memory storing computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least perform the computer-implemented method of any preceding claim.
25. A computer program comprising program code instructions which, when executed by a processor of an apparatus, cause the apparatus to perform the computer-implemented method of any of claims 1-22.