JP7337667B2 - Puncture support device - Google Patents

Description

The embodiments disclosed in this specification and the like relate to a puncture assist device.

2. Description of the Related Art Conventionally, in order to easily observe the inside of a subject's body, an ultrasonic diagnostic apparatus has been widely used that transmits ultrasonic waves from the body surface into the body and displays an ultrasonic image based on the reflected waves. For example, since an ultrasonic diagnostic apparatus can display an ultrasonic image on a monitor in substantially real time, biopsy, radio frequency ablation (RFA), and irreversible electroporation (IRE) are used. It is used when puncturing is performed, such as for medical treatment.

In puncture using such a puncture needle, a puncture plan is made based on images taken in advance such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging), and the puncture needle is guided by ultrasound images during treatment. inserted. Here, when performing puncture using an ultrasound image as a guide, a needle adapter that fixes the angle of the puncture needle may be used so that the puncture needle is visualized in the ultrasound image. By using a needle adapter, the puncture needle is present on the same plane as the ultrasonic image, which is useful when it is difficult to visually recognize the needle during puncture.

In addition, it is sometimes difficult to visualize the affected area with ultrasound when guiding puncture using ultrasound images. A Fusion function may be used to match the angles and positions of both planes for display.

Moreover, in recent years, a method of inserting a plurality of needles simultaneously has become widespread in puncture for the purpose of treatment. For example, a plurality of puncture needles are used when the affected area is large and cannot be treated with one treatment, or when a plurality of needles are required in principle depending on the treatment method. One example of puncture using a plurality of puncture needles is irreversible electroporation, also called Nanoknife (registered trademark). Since the nanoknife uses a different needle for each electrode, at least two simultaneous punctures are required, and six simultaneous punctures may be performed. The nanoknife treats a target between electrodes by irradiating a pulse wave for about 30 minutes to 1 hour after puncturing. Such multiple punctures are difficult to perform without prior planning due to the complexity of the procedure, and puncture planning is performed in advance using CT, MRI, and ultrasound images. Puncture is performed according to this puncture plan, guided by real-time ultrasound images.

JP 2019-13788 A

One of the problems to be solved by the embodiments disclosed in this specification and the like is to reduce the load associated with puncturing. However, the problems to be solved by the embodiments disclosed in this specification and the like are not limited to the above problems. A problem corresponding to each effect of each configuration shown in the embodiments described later can be positioned as another problem to be solved by the embodiments disclosed in this specification and the like.

A puncture assistance device according to an embodiment includes an acquisition unit, a calculation unit, and a control unit. The acquisition unit acquires information on the puncture position for the subject set based on the puncture plan. The calculation unit calculates a difference between the position information of the subject in the puncture position information and the position information of the subject based on real-time ultrasonic images acquired from the subject. The control unit controls a position of a robot arm holding a puncture needle to be inserted into the subject based on the puncture plan so as to adjust the position using the difference.

FIG. 1 is a diagram schematically showing an example of a puncture support system according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of the ultrasonic diagnostic apparatus according to the first embodiment. FIG. 3 is a block diagram showing an example of the configuration of the robot arm device according to the first embodiment. FIG. 4 is a diagram illustrating an example of an arm that holds a puncture needle according to the first embodiment; FIG. 5 is a diagram illustrating an example of an arm that holds an ultrasonic probe according to the first embodiment; FIG. 6 is a flowchart for explaining the procedure of the procedure according to the first embodiment. FIG. 7 is a diagram schematically showing an example of a puncture plan according to the first embodiment; FIG. 8 is a diagram for explaining an example of the Fusion function according to the first embodiment; FIG. 9 is a diagram showing an example of specifying a position via an ultrasound image according to the first embodiment. FIG. 10 is a diagram illustrating an example of processing by the display control function according to the first embodiment; FIG. 11 is a diagram illustrating an example of processing by the display control function according to the first embodiment; FIG. 12 is a flowchart for explaining the processing procedure of the ultrasonic diagnostic apparatus according to the first embodiment. FIG. 13 is a flowchart for explaining the processing procedure of the ultrasonic diagnostic apparatus according to the first embodiment.

Hereinafter, embodiments of a puncture assisting device according to the present application will be described in detail with reference to the accompanying drawings. Note that the puncture assistance device according to the present application is not limited to the embodiments shown below. In addition, in the following description, common reference numerals are assigned to similar components, and duplicate descriptions are omitted.

(First embodiment)
First, a puncture assistance system including a puncture assistance device according to the present application will be described. The puncture assistance system includes an ultrasonic diagnostic device and a robot arm device, and adjusts a puncture needle held by the robot arm device under control of the puncture assistance device. FIG. 1 is a diagram schematically showing an example of a puncture support system 1000 according to the first embodiment. For example, as shown in the figure, a puncture support system 1000 includes an ultrasonic diagnostic device (US device) 1 and a robot arm device 200, and supports puncture of a subject.

The ultrasonic diagnostic apparatus 100 has an ultrasonic probe 12 and generates an ultrasonic image based on ultrasonic waves transmitted and received by the ultrasonic probe 12 . For example, the ultrasonic diagnostic apparatus 100 generates an ultrasonic image of the puncture target site of the subject scanned by the ultrasonic probe 12 .

The robot arm device 200 has, for example, a robot arm control device 21, an arm 22a, an arm 22b, and an arm 22c. The arm 22a holds the puncture needle 31 and changes its position or moves the puncture needle 31 in the insertion direction (longitudinal direction of the puncture needle 31) under the control of the robot arm control device 21. The arm 22b holds the puncture needle 32 and changes its position or moves the puncture needle 31 in the insertion direction (longitudinal direction of the puncture needle 31) under the control of the robot arm control device 21. The arm 22 c holds the ultrasonic probe 12 and changes its position based on the control by the robot arm control device 21 to execute an ultrasonic scan on the subject.

The robot arm control device 21 adjusts the positions of the puncture needle 31, the puncture needle 32, and the ultrasonic probe 12 with respect to the subject by moving the positions of the arms 22a, 22b, and 22c. Here, the robot arm control device 21 can control the positions of the arms 22a, 22b, and 22c based on control signals received from the puncture support device. That is, the robot arm control device 21 moves the arms 22a, 22b, and 22c based on the amount of movement of the arms included in the control signal received from the puncture support device. The positional relationship between the arms 22a, 22b, and 22c and the subject can be grasped based on the positional information acquired by the position sensor 61b attached to the subject.

The puncture assistance device transmits a control signal to the robot arm device 200 to adjust the positions of the arms 22a, 22b, and 22c of the robot arm device 200, thereby assisting the puncture of the subject. Here, the puncture assistance device according to the present application can reduce the load related to puncture. Specifically, the puncture support device makes it possible to reduce the burden on the operator when performing puncture according to a puncture plan.

As described above, when performing a puncture, a puncture plan is made based on a three-dimensional image such as CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) captured in advance, and the puncture is performed according to this plan. is executed. However, in the puncture planning, since the organs in the subject are treated as rigid bodies, deviations occur due to deformation of the organs due to changes in the subject's posture and respiration during puncture, and the insertion of the puncture needle does not necessarily go as planned. Not always possible. Therefore, it is necessary to correct this deviation, but it is difficult to accurately grasp the correction direction and correction amount from the two-dimensional information of the ultrasonic image acquired in real time. In addition, in order to acquire real-time ultrasound images, it is difficult to adjust the position and angle of the ultrasound probe due to the restriction imposed by the position of the puncture needle. Also, by using the Fusion function, it is possible to calculate the correction direction and correction amount, but it is necessary to grasp how the calculated correction direction and correction amount should be applied to the position and angle of the puncture needle. It's not easy either.

Moreover, when performing multiple punctures, the operator is required to have high skill. In such a case, for example, the operator must be able to three-dimensionally grasp the relationship between a plurality of puncture needles placed in the body and the organ, and must be able to visually recognize the puncture needle and place it in an appropriate position for puncture. It is required to arrange a sound wave probe and perform a scan.

In this way, since puncturing places various loads on the operator, the puncture support device according to the present application controls the robot arm device 200 to place the puncture needle at the puncture position according to the puncture plan. In addition, by adjusting the position of the puncture needle based on real-time ultrasound images, it is possible to reduce the load associated with puncture. Furthermore, the puncture assistance device can detect body movement of the subject based on information acquired by the biological signal sensor 62 and control the robot arm device 200 in accordance with the detected body movement.

Details of the first embodiment of the present application will be described below. In the first embodiment, a case in which the puncture assisting apparatus according to the present application is included in an ultrasonic diagnostic apparatus will be described as an example, but the embodiment is not limited to this. That is, the various processes executed by the ultrasonic diagnostic apparatus described below may be executed by any device in the puncture support system 1000 described above. Further, a puncture assistance apparatus may be further connected to the puncture assistance system 1000 described above, and the connected puncture assistance apparatus may execute various processes executed by the ultrasonic diagnostic apparatus described below.

FIG. 2 is a block diagram showing an example of the configuration of the ultrasonic diagnostic apparatus 100 according to the first embodiment. As shown in FIG. 2, the ultrasonic diagnostic apparatus 100 according to this embodiment includes an ultrasonic probe 12, a display 13, an input interface 14, and an apparatus body 15. The ultrasonic probe 12 and the display 13 , the input interface 14 is communicatively connected to the device body 15 . Further, the ultrasonic diagnostic apparatus 100 according to the present embodiment is further communicably connected to the robot arm apparatus 200 .

The ultrasonic diagnostic apparatus 100 is also connected to a position sensor 61a, a position sensor 61b, a transmitter 61c, and a biological signal sensor 62. FIG.

The position sensor 61 a is, for example, a magnetic sensor and attached to the ultrasonic probe 12 . The position sensor 61b is, for example, a magnetic sensor and attached to the subject. The transmitter 61c is a device that forms a magnetic field outward from its center, and is arranged at an arbitrary position near the device main body 15 . The position sensor 61a, the position sensor 61b, and the transmitter 61c are a position detection system for detecting position information of the ultrasonic probe 12 and position information of the subject.

The position sensor 61a detects the intensity and gradient of the three-dimensional magnetic field produced by the transmitter 61c. Then, the position sensor 61a calculates the position (coordinates and angles) of its own device in the space with the transmitter 61c as the origin based on the detected magnetic field information, and transmits the calculated position to the device body 15. FIG. Here, the position sensor 61a transmits the three-dimensional coordinates and angle of its own device to the device main body 15 as three-dimensional position information of the ultrasonic probe 12 . Thereby, the device main body 15 can calculate the position of the ultrasonic image in the space with the transmitter 61c as the origin.

A plurality of position sensors 61b are arranged on the subject and detect the strength and inclination of the three-dimensional magnetic field formed by the transmitter 61c. Then, the position sensor 61b calculates the position (coordinates and angles) of its own device in the space with the transmitter 61c as the origin based on the detected magnetic field information, and transmits the calculated position to the device body 15. FIG. Here, the position sensor 61b transmits the three-dimensional coordinates and angle of its own device to the device body 15 as three-dimensional position information of the subject. The device main body 15 receives three-dimensional positional information of the subject (three-dimensional positional information of the position of the subject where each position sensor 61b is attached) received from each of the plurality of position sensors 61b, and pre-input shape of the subject. and size information, each position of the subject in the space with the transmitter 61c as the origin can be calculated.

Note that this embodiment can be applied even when the position information of the ultrasonic probe 12 and the subject is acquired by a system other than the position detection system described above. For example, in the present embodiment, a gyro sensor, an acceleration sensor, or the like may be used to obtain the position information of the ultrasonic probe 12 and the subject.

The biological signal sensor 62 is a sensor that detects respiration of the subject, a sensor that detects an ECG (electrocardiogram), a body surface sensor that detects movement of the body surface, or the like. The biomedical signal sensor 62 transmits respiratory information, ECG information, and body surface movement information acquired from the subject to the apparatus main body 15 .

The ultrasonic probe 12 is connected to a transmission/reception circuit 151 included in the device body 15 . The ultrasonic probe 12 has, for example, a plurality of piezoelectric transducers in its probe body, and these piezoelectric transducers generate ultrasonic waves based on drive signals supplied from the transmission/reception circuit 151 . The ultrasonic probe 12 also receives reflected waves from the subject and converts them into electrical signals. Further, the ultrasonic probe 12 has, in the probe main body, a matching layer provided on the piezoelectric vibrator, a backing material for preventing ultrasonic waves from propagating backward from the piezoelectric vibrator, and the like. The ultrasonic probe 12 is detachably connected to the device body 15 . For example, the ultrasonic probe 12 is a sector type, linear type or convex type ultrasonic probe.

When ultrasonic waves are transmitted from the ultrasonic probe 12 to the subject, the transmitted ultrasonic waves are successively reflected by discontinuous surfaces of acoustic impedance in the body tissue of the subject, and the ultrasonic probe 12 emits a reflected wave signal. received by a plurality of piezoelectric vibrators. The amplitude of the received reflected wave signal depends on the difference in acoustic impedance at the discontinuity from which the ultrasonic waves are reflected. When the transmitted ultrasonic pulse is reflected by the moving blood flow or the surface of the heart wall, the reflected wave signal depends on the velocity component of the moving object in the ultrasonic transmission direction due to the Doppler effect. subject to frequency shifts.

In this embodiment, even when the subject is scanned two-dimensionally by the ultrasonic probe 12, which is a one-dimensional ultrasonic probe in which a plurality of piezoelectric transducers are arranged in a line, the one-dimensional ultrasonic probe The ultrasonic probe 12 mechanically oscillates a plurality of piezoelectric transducers or the ultrasonic probe 12, which is a two-dimensional ultrasonic probe in which a plurality of piezoelectric transducers are arranged two-dimensionally in a grid pattern, vibrates the subject into three It is applicable even when scanning in dimensions.

The display 13 displays a GUI (Graphical User Interface) for the operator of the ultrasonic diagnostic apparatus 100 to input various setting requests using the input interface 14, and displays ultrasonic images and the like generated in the apparatus main body 15. to display. Further, the display 13 displays various messages and display information in order to notify the operator of the processing status and processing results of the device body 15 . The display 13 also has a speaker and can output sound.

The input interface 14 includes a trackball for setting a predetermined position (for example, a region of interest, etc.), a switch button, a mouse, a keyboard, a touch pad for performing input operations by touching the operation surface, a display screen and a touch pad. It is realized by a touch monitor integrated with and a non-contact input circuit using an optical sensor, an audio input circuit, and the like. The input interface 14 is connected to a processing circuit 155 to be described later, converts an input operation received from an operator into an electric signal, and outputs the electric signal to the processing circuit 155 . It should be noted that the input interface 14 in this specification is not limited to having physical operation parts such as a mouse and a keyboard. For example, the input interface includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to the processing circuit 155 .

The device main body 15 has a transmission/reception circuit 151 , a B-mode processing circuit 152 , a Doppler processing circuit 153 , a memory 154 , a processing circuit 155 and a communication interface 156 . In the ultrasonic diagnostic apparatus 1 shown in FIG. 2, each processing function is stored in the memory 154 in the form of a computer-executable program. The transmission/reception circuit 151, the B-mode processing circuit 152, the Doppler processing circuit 153, and the processing circuit 155 are processors that read and execute programs from the memory 154 to implement functions corresponding to each program. In other words, each circuit with each program read has a function corresponding to the read program.

The transmission/reception circuit 151 has a pulse generator, a transmission delay circuit, a pulser, and the like, and supplies drive signals to the ultrasonic probe 12 . A pulse generator repeatedly generates rate pulses at a predetermined rate frequency to form a transmitted ultrasound wave. In addition, the transmission delay circuit focuses the ultrasonic waves generated from the ultrasonic probe 12 into a beam, and the pulse generator generates a delay time for each piezoelectric transducer necessary for determining the transmission directivity. given for each rate pulse. Also, the pulsar applies a driving signal (driving pulse) to the ultrasonic probe 12 at a timing based on the rate pulse. That is, the transmission delay circuit arbitrarily adjusts the transmission direction of the ultrasonic waves transmitted from the piezoelectric transducer surface by changing the delay time given to each rate pulse.

The transmitting/receiving circuit 151 has a function capable of instantaneously changing the transmission frequency, the transmission drive voltage, etc. in order to execute a predetermined scan sequence based on instructions from the processing circuit 155, which will be described later. In particular, the change of the transmission drive voltage is realized by a linear amplifier type oscillator circuit capable of instantaneously switching the value or by a mechanism for electrically switching between a plurality of power supply units.

Further, the transmission/reception circuit 151 has a preamplifier, an A/D (Analog/Digital) converter, a reception delay circuit, an adder, etc., and performs various processing on the reflected wave signal received by the ultrasonic probe 12 to Generate wave data. A preamplifier amplifies the reflected wave signal for each channel. The A/D converter A/D converts the amplified reflected wave signal. A reception delay circuit provides a delay time required to determine reception directivity. The adder adds the reflected wave signals processed by the reception delay circuit to generate reflected wave data. The addition processing of the adder emphasizes the reflection component from the direction corresponding to the reception directivity of the reflected wave signal, and the reception directivity and the transmission directivity form a comprehensive beam for ultrasonic wave transmission/reception.

The B-mode processing circuit 152 receives the reflected wave data from the transmitting/receiving circuit 151, performs logarithmic amplification, envelope detection processing, etc., and generates data (B-mode data) in which the signal strength is represented by brightness of luminance. .

The Doppler processing circuit 153 frequency-analyzes velocity information from the reflected wave data received from the transmission/reception circuit 151, extracts blood flow, tissue, and contrast agent echo components due to the Doppler effect, and obtains moving body information such as velocity, dispersion, and power. Data extracted from multiple points (Doppler data) is generated. For example, the moving body is fluid such as blood flowing in blood vessels and lymph fluid flowing in lymphatic vessels.

The B-mode processing circuit 152 and the Doppler processing circuit 153 can process both two-dimensional reflected wave data and three-dimensional reflected wave data. That is, the B-mode processing circuit 152 generates two-dimensional B-mode data from the two-dimensional reflected wave data, and generates three-dimensional B-mode data from the three-dimensional reflected wave data. The Doppler processing circuit 153 also generates two-dimensional Doppler data from the two-dimensional reflected wave data, and generates three-dimensional Doppler data from the three-dimensional reflected wave data. The three-dimensional B-mode data is data to which luminance values corresponding to the reflection intensity of the reflection source located at each of a plurality of points (sample points) set on each scanning line in the three-dimensional scanning range are assigned. In the three-dimensional Doppler data, each of a plurality of points (sample points) set on each scanning line in the three-dimensional scanning range is assigned a brightness value corresponding to the value of blood flow information (velocity, dispersion, power). data.

The B-mode processing circuit 152 and the Doppler processing circuit 153 combine a plurality of two-dimensional reflected wave data to generate three-dimensional reflected wave data, and generate three-dimensional data from the generated reflected wave data. can also For example, the B-mode processing circuit 152 combines a plurality of two-dimensional reflected wave data to generate three-dimensional reflected wave data, and generates three-dimensional B-mode data from the generated three-dimensional reflected wave data. The Doppler processing circuit 153 also generates three-dimensional reflected wave data by synthesizing a plurality of two-dimensional reflected wave data, and generates three-dimensional Doppler data from the generated three-dimensional reflected wave data.

The memory 154 stores image data for display generated by the processing circuit 155 . The memory 154 can also store data generated by the B-mode processing circuit 152 and the Doppler processing circuit 153 . In addition, the memory 154 stores control programs for transmitting and receiving ultrasonic waves, image processing, and display processing, diagnostic information (for example, patient ID, doctor's findings, etc.), various data such as diagnostic protocols and various body marks. do. In addition, the memory 154 stores various information received from the position sensor 61a, the position sensor 61b, and the biological signal sensor 62. FIG.

The communication interface 156 is connected to the processing circuit 155 and controls communications between the ultrasonic diagnostic apparatus 100 and each apparatus. Specifically, the communication interface 156 receives various information from each device and outputs the received information to the processing circuit 155 . For example, the communication interface 156 is implemented by a network card, network adapter, NIC (Network Interface Controller), or the like. For example, the communication interface 156 transmits the control signal received from the processing circuit 155 to the robot arm device 200 .

The processing circuitry 155 controls the entire processing of the ultrasonic diagnostic apparatus 100 . Specifically, the processing circuit 155 reads from the memory 154 programs corresponding to the control function 155a, the image generation function 155b, the acquisition function 155c, the calculation function 155d, the arm control function 155e, and the display control function 156f shown in FIG. By executing it, various processes are performed. For example, the processing circuit 155 is a processor that reads out each program from the memory 154 and executes it to implement the function corresponding to each program. In other words, the processing circuit 155 with each program read has each function shown in the processing circuit 155 of FIG.

Here, the control function 155a is an example of an acquisition unit. Acquisition function 155c is an example of a biometric information acquisition unit. Also, the calculation function 155d is an example of a calculation unit. Also, the arm control function 155e is an example of a control unit. Also, the display control function 156f is an example of a display control unit. In this embodiment, a single processing circuit 155 is assumed to implement each processing function described below. may implement the function by executing the program.

The control function 155a controls the transmission/reception circuit 151, the B-mode processing circuit 152, and the Doppler processing circuit based on various setting requests input by the operator via the input interface 14 and various control programs and various data read from the memory 154. 153 processing is controlled.

In addition, the control function 155a acquires information on a preplanned puncture plan via the communication interface 156. FIG. For example, the control function 155a acquires a puncture plan based on three-dimensional medical image data acquired by a CT device, MRI device, or ultrasonic diagnostic device. That is, the control function 155a acquires information on the puncture position for the subject set based on the puncture plan.

The image generation function 155b generates ultrasound image data from the data generated by the B-mode processing circuit 152 and the Doppler processing circuit 153. FIG. That is, the image generation function 155b generates B-mode image data representing the intensity of the reflected wave by luminance from the two-dimensional B-mode data generated by the B-mode processing circuit 152. FIG. The B-mode image data is data representing the tissue shape in the ultrasonically scanned region. Further, the image generation function 155b generates Doppler image data representing moving body information from the two-dimensional Doppler data generated by the Doppler processing circuit 153. FIG. Doppler image data is velocity image data, variance image data, power image data, or image data combining these. The Doppler image data is data representing fluid information about fluid flowing within the ultrasonically scanned region.

Here, the image generating function 155b generally converts (scan-converts) a scanning line signal train of ultrasonic scanning into a scanning line signal train of a video format typified by television, etc., and converts the ultrasonic wave for display. Generate image data. Specifically, the image generating function 155b performs coordinate conversion according to the scanning mode of ultrasonic waves by the ultrasonic probe 12, thereby generating ultrasonic image data for display. In addition, the image generation function 155b performs various image processing other than scan conversion, such as image processing (smoothing processing) for regenerating an average brightness image using a plurality of image frames after scan conversion, Image processing (edge enhancement processing) or the like using a differential filter is performed within the image. Further, the image generating function 155b synthesizes character information of various parameters, scales, body marks, etc. with the ultrasonic image data.

That is, the B-mode data and Doppler data are ultrasound image data before scan conversion processing, and the data generated by the image generation function 155b are ultrasound image data for display after scan conversion processing. B-mode data and Doppler data are also called raw data.

Further, the image generation function 155b performs coordinate transformation on the three-dimensional B-mode data generated by the B-mode processing circuit 152 to generate three-dimensional B-mode image data. The image generation function 155b also performs coordinate transformation on the three-dimensional Doppler data generated by the Doppler processing circuit 153 to generate three-dimensional Doppler image data. That is, the three-dimensional B-mode data and three-dimensional Doppler data are volume data before scan conversion processing, and the three-dimensional B-mode image data and three-dimensional Doppler image data are volume data after scan conversion. .

Furthermore, the image generation function 155b can perform rendering processing on volume data in order to generate various types of two-dimensional image data for displaying the volume data on the display 13. FIG.

The acquisition function 155c acquires the three-dimensional position information of the ultrasonic probe 12 acquired by the position sensor 61a. Also, the acquisition function 155c acquires the three-dimensional position information of the subject acquired by the plurality of position sensors 61b. In addition, the acquisition function 155c acquires respiratory information, an electrocardiogram, body surface movement information, and the like acquired by the biological signal sensor 62 .

The calculation function 155d calculates the difference between the position information of the subject when planning the puncture and the position information of the subject when performing the puncture. Specifically, the calculation function 155d calculates the deviation between the positional relationship between the body surface and the organ when planning the puncture and the positional relationship between the body surface and the organ when performing the puncture. For example, the calculation function 155d calculates the difference between the position information of the subject in the puncture position information and the position information of the subject based on real-time ultrasound images acquired from the subject.

The arm control function 155e controls the robot arm device 200 that holds the puncture needle based on the position information deviation calculated by the calculation function 155d. Specifically, the arm control function 155e first transmits a control signal to the robot arm device 200 so as to place the puncture needle at the position based on the position information at the time of planning the puncture. The arm control function 155e controls the robot arm device 200 so as to adjust the position and angle of the puncture needle according to the deviation calculated by the calculation function 155d. For example, the arm control function 155e sends a control signal to the robot arm device 200 to adjust the position of the arm holding the puncture needle to be inserted into the subject based on the puncture plan using the difference. to send.

The display control function 155f controls the display 13 to display ultrasonic image data for display (hereinafter also referred to as an ultrasonic image) stored in the memory 154 . Further, the display control function 155f controls to display the processing result on the display 13. FIG. The display control function 155f controls display of various information on the touch monitor of the input interface 14. FIG.

FIG. 3 is a block diagram showing an example of the configuration of the robot arm device 200 according to the first embodiment. As shown in FIG. 3, the robot arm device 200 according to this embodiment has a robot arm control device 21, an arm 22a, an arm 22b, and an arm 22c. The robot arm device 200 also includes position sensors 23a, 23b and 23c on the arms 22a, 22b and 22c. Further, the robot arm device 200 according to this embodiment is connected to the ultrasonic diagnostic device 100 so as to be able to communicate therewith. The robot arm device 200 is also connected to the biosignal sensor 62 .

Although FIG. 3 shows the robot arm device 200 having three arms 22a, 22b, and 22c, the embodiment is not limited to this. and arms for holding the ultrasonic probe 12, the number of arms is arbitrary. For example, it may have one arm or three or more arms that hold the puncture needle, or it may have two or more arms that hold the ultrasonic probe 12 .

Arm 22 a holds puncture needle 31 and moves puncture needle 31 under the control of robot arm control device 21 . Specifically, the arm 22a has a holder that holds the puncture needle 31 and a drive mechanism that drives the arm. The arm 22a moves the puncture needle 31 held on the distal end side to a desired position by moving a plurality of movable parts having drive mechanisms such as motors and actuators according to control received from the robot arm control device 21. Let Here, the arm 22a is configured so that the puncture needle 31 can be arranged at any three-dimensional position with respect to the subject.

Arm 22 b holds puncture needle 32 and moves puncture needle 32 under the control of robot arm control device 21 . Specifically, the arm 22b has a holder that holds the puncture needle 32 and a drive mechanism that drives the arm. The arm 22b moves the puncture needle 32 held on the distal end side to a desired position by moving a plurality of movable parts having drive mechanisms such as motors and actuators according to control received from the robot arm control device 21. Let Here, the arm 22b is configured so that the puncture needle 32 can be arranged at any three-dimensional position with respect to the subject.

Here, the arms 22a and 22b can also move the puncture needles 31 and 32 in the longitudinal direction of the needles. That is, the arms 22a and 22b can be configured to automatically insert and remove the puncture needle under the control of the robot arm control device 21. FIG. FIG. 4 is a diagram illustrating an example of an arm that holds a puncture needle according to the first embodiment; For example, the arm 22a has a rubber roller 221 on the tip side of the arm, as shown in FIG. The rubber rollers 221 are connected to a driving mechanism such as a motor or an actuator, hold the puncture needle 31 between a plurality of rollers, and move the puncture needle 31 in the longitudinal direction of the needle by rotating the plurality of rollers. .

Here, the rubber roller 221 holds the puncture needle 31 so that the puncture needle 31 does not move in any direction other than the longitudinal direction of the needle. Also, the rubber roller 221 can move the puncture needle 31 in centimeter units, for example. That is, the rotation of the rubber roller 221 is controlled so as to move the puncture needle 31 in centimeter units.

Here, the rubber roller 221 can further automatically release the held puncture needle 31 . For example, the rubber roller 221 has a pressure sensor, and automatically releases the held puncture needle 31 when a pressure equal to or greater than a threshold value is applied to any of the rollers. As a result, for example, safety can be ensured against the subject's body movement and the case where an external force is applied to the arm 22a.

Also in the arm 22b, similarly to the example of the arm 22a described above, by having a rubber roller on the distal end side of the arm, the puncture needle can be automatically inserted and removed. The arms 22a and 22b can adjust the holding force of the puncture needle via the input interface 212. FIG. For example, the operator can manually insert the puncture needles 31 and 32 by weakening the holding force of the puncture needles via the input interface 212 .

The arm 22 c holds the ultrasonic probe 12 and moves the ultrasonic probe 12 under the control of the robot arm control device 21 . Specifically, the arm 22c has a holder that holds the ultrasonic probe 12 and a drive mechanism that drives the arm. The arm 22c moves the ultrasonic probe 12 held on the distal end side to a desired position by moving a plurality of movable parts having drive mechanisms such as motors and actuators according to control received from the robot arm control device 21. move. Here, the arm 22c is configured so that the ultrasonic probe 12 can be placed at any three-dimensional position with respect to the subject.

FIG. 5 is a diagram showing an example of an arm that holds the ultrasonic probe 12 according to the first embodiment. For example, as shown in FIG. 5, the arm 22c holds the ultrasonic probe 12 on the distal end side. Under the control of the robot arm controller 21, the arm drive mechanism drives the arm 22c to move the ultrasonic probe 12 in the X-axis direction, the Y-axis direction, and the Z-axis direction. Further, the arm 22c rotates the ultrasonic probe 12 in the directions of the arrows 41 to 43 by driving the drive mechanism of the holding portion under the control of the robot arm control device 21, thereby changing the orientation of the ultrasonic probe 12. change.

Here, the holding portion of the arm 22c can further automatically release the held ultrasonic probe 12 . For example, the holding portion of the arm 22c has a pressure sensor, and automatically releases the held ultrasonic probe 12 when a pressure equal to or higher than a threshold is applied to the holding portion. As a result, for example, when the ultrasonic scanning condition is poor, the operator can easily remove the ultrasonic probe 12 by grasping the ultrasonic probe 12 and applying force in any direction.

Also, the holding portion of the arm 22c can be further configured to automatically hold the ultrasonic probe 12 . Specifically, when the operator brings the ultrasonic probe 12 closer to the holding part, the holding part of the arm 22c grabs and holds the ultrasonic probe 12 . In such a case, for example, the holding portion of the arm 22c is equipped with a non-contact sensor or the like, and the control function 213a moves the holding portion of the arm 22c according to the signal from the non-contact sensor.

Returning to FIG. 3, position sensor 23a detects the strength and gradient of the three-dimensional magnetic field formed by transmitter 61c. Then, the position sensor 23 a calculates the position (coordinates and angles) of its own device in the space with the transmitter 61 c as the origin based on the detected magnetic field information, and transmits the calculated position to the robot arm control device 21 . Here, the position sensor 23a transmits the three-dimensional coordinates and angle of its own device to the robot arm control device 21 as three-dimensional position information of the arm 22a. The robot arm control device 21 can calculate the position of the arm 22a in the space with the transmitter 61c as the origin from the position where the position sensor 23a is attached and the information on the shape and size of the arm 22a that has been input in advance. .

Similarly, position sensor 23b and position sensor 23c detect the strength and gradient of the three-dimensional magnetic field produced by transmitter 61c, respectively. Then, the position sensors 23b and 23c calculate the positions (coordinates and angles) of their own devices in the space with the transmitter 61c as the origin based on the detected magnetic field information, and transmit the calculated positions to the robot arm control device. 21 respectively. Based on the positions where the position sensors 23b and 23c are attached and information on the shapes and sizes of the arms 22b and 22c previously input, the robot arm control device 21 determines the positions of the arms 22b and 22c in the space with the transmitter 61c as the origin. The position of arm 22c can be calculated.

The puncture needle 31 and the puncture needle 32 are, for example, electromagnetic needles that generate radio waves, and are connected to a therapeutic device that controls the output of the radio waves generated by the puncture needles 31 and 32 . This therapeutic device can monitor the temperature of the puncture needles 31 and 32, the output of radio waves, and the impedance of the cauterization region. Proceed with the RFA used.

The puncture needle 31 and the puncture needle 32 are, for example, electrode needles that apply current to the tissue to be treated, and are connected to a therapeutic device that controls the output of the current generated by the puncture needle 31 and the puncture needle 32 . Here, this treatment apparatus applies current to the tissue to be treated between the puncture needles 31 and 32 by passing an electric current between the puncture needles 31 and 32 for treatment.

The robot arm control device 21 has a communication interface 211 , an input interface 212 and a processing circuit 213 .

The communication interface 211 is connected to the processing circuit 213 and controls communication between the robot arm control device 21 and each device. Specifically, the communication interface 211 receives various types of information from each device and outputs the received information to the processing circuit 213 . For example, the communication interface 211 is implemented by a network card, network adapter, NIC (Network Interface Controller), or the like. For example, the communication interface 211 receives a control signal from the ultrasonic diagnostic apparatus 100 and outputs the received control signal to the processing circuit 213 .

The input interface 212 includes a trackball, a switch button, a mouse, a keyboard, a touch pad for performing input operations by touching the operation surface, a touch monitor in which the display screen and the touch pad are integrated, an optical It is realized by a non-contact input circuit using a sensor, an audio input circuit, and the like. The input interface 212 is connected to the processing circuit 213 , converts an input operation received from an operator into an electric signal, and outputs the electric signal to the processing circuit 213 . It should be noted that the input interface 212 in this specification is not limited to having physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to the processing circuit 213 is also included in the input interface.

A processing circuit 213 controls the entire processing of the robot arm control device 21 . Specifically, the processing circuit 213 performs various processes by reading and executing programs corresponding to the control function 213a, the calculation function 213b, and the acquisition function 213c shown in FIG. 3 from a memory (not shown). For example, the processing circuit 213 is a processor that implements functions corresponding to each program by reading each program from a memory (not shown) and executing the program. In other words, the processing circuit 213 with each program read has each function shown in the processing circuit 213 of FIG.

In this embodiment, the single processing circuit 213 is assumed to implement each processing function described below. may implement the function by executing the program.

The control function 213a controls driving of the arms 22a, 22b and 22c by transmitting control signals to the drive mechanisms of the arms 22a, 22b and 22c. Specifically, the control function 213a moves each arm by transmitting a control signal based on the arm movement information (movement direction and movement amount) received from the arithmetic function 213b to the drive mechanism. Further, the control function 213a moves each arm by transmitting a control signal based on arm movement information received from the ultrasonic diagnostic apparatus 100 via the communication interface 211 to the drive mechanism. The arm movement information received from the ultrasonic diagnostic apparatus 100 will be described in detail later.

The calculation function 213b calculates arm movement information from the current positions (three-dimensional coordinates) of the arms 22a, 22b, and 22c and the movement destination positions (three-dimensional coordinates), and calculates the calculated arm movement information. to the control function 213a. For example, the arithmetic function 213b calculates the movement direction and the movement amount from the current position of each arm and the position of the movement destination input via the input interface 212, and transmits them to the control function 213a. Further, for example, the calculation function 213b calculates the movement direction and the movement amount from the current position of each arm and the position of the movement destination received from the ultrasonic diagnostic apparatus 100 via the communication interface 211, and calculates the movement direction and the movement amount. You can also send to

The acquisition function 213c acquires the three-dimensional position information of the arm 22a acquired by the position sensor 23a. Also, the acquisition function 213c acquires the three-dimensional position information of the arm 22b acquired by the position sensor 23b. Also, the acquisition function 213c acquires the three-dimensional position information of the arm 22c acquired by the position sensor 23c. Also, the acquisition function 213c acquires respiratory information, an electrocardiogram, movement information of the body surface, and the like acquired by the biological signal sensor 62 . The acquisition function 213c transmits the acquired three-dimensional position information, respiratory information, electrocardiogram, movement information of the body surface, etc. to the control function 213a and the calculation function 213b.

The overall configuration of the puncture support system 1000 according to the first embodiment has been described above. Next, the details of the puncture support device (ultrasonic diagnostic device 100) according to the first embodiment will be described. Here, first, the procedure of the procedure using the puncture assistance device of the present application will be described with reference to FIG. 6 . FIG. 6 is a flowchart for explaining the procedure of the procedure according to the first embodiment. Here, steps S101 to S102 in FIG. 6 are procedures performed when making a puncture plan, and steps S103 to S109 are procedures performed when treatment by puncture is actually performed.

As shown in FIG. 6, in a procedure using the puncture assist device of the present application, for example, the operator first uses a CT device, an MRI device, an ultrasonic diagnostic device 100, or the like to obtain three-dimensional image data (volume data). ) is collected (step S101). Then, the operator uses the collected three-dimensional image data to plan the puncture position (step S102).

FIG. 7 is a diagram schematically showing an example of a puncture plan according to the first embodiment; For example, the operator observes three-dimensional image data acquired using a CT device, an MRI device, the ultrasonic diagnostic device 100, or the like, and identifies the position of the target to be treated. Then, the operator determines the positions and the number of puncture needles to be inserted in consideration of the shape and size of the target and the positions of surrounding organs and bones. For example, the operator determines the insertion position and angle of each puncture needle on the body surface so that the target can be treated appropriately while avoiding blood vessels, various organs, and bones. . That is, in the puncture plan, the position on the body surface where the puncture needle is to be inserted and the insertion angle of the puncture needle are determined at the coordinates of the volume data.

After the puncture plan is made as described above, the operator operates the robot arm device 200 to set the puncture needle at the planned puncture position (step S103), and attaches the ultrasonic probe 12 to the robot arm device 200. is held to acquire an ultrasound image (step S104).

The ultrasonic diagnostic apparatus 100 checks whether there is a difference between the position of the subject based on the acquired ultrasonic image and the planned position (step S105), and determines whether there is any positional deviation. is determined (step S106). Here, if there is positional deviation (Yes at step S106), the ultrasonic diagnostic apparatus 100 corrects the positional deviation (step S107). After correcting the positional deviation, or when there is no positional deviation (No at step S106), the operator performs puncture (step S108) and performs treatment (step S109).

As described above, the puncture assisting apparatus (ultrasonic diagnostic apparatus 100) according to the first embodiment uses an ultrasound image to detect the positional deviation between when planning the puncture and when performing the puncture, and detects the detected positional deviation. fix it. Here, the ultrasound diagnostic apparatus 100 uses the Fusion function in order to detect the positional deviation between when the puncture is planned and when the puncture is performed.

Specifically, the control function 155a of the ultrasonic diagnostic apparatus 100 performs registration between the ultrasonic image and the volume data used for the puncture planning. More specifically, the control function 155a controls the three-dimensional space in which the ultrasound images were acquired (the first coordinate space formed by the transmitter 61c) and the three-dimensional space in which the volume data was acquired (the second coordinate space). Determine the correspondence of the coordinates with That is, the control function 155a determines the position (coordinates) in the second coordinate space corresponding to the position (coordinates) of the ultrasound image in the first coordinate space.

For example, the control function 155a aligns the part included in the ultrasound image and the corresponding part in the volume data to substantially the same position, and adjusts the ultrasound image in the coordinate space (second coordinate space) of the volume data at that time. determine the position of As a result, as shown in FIG. 8, the ultrasonic diagnostic apparatus 100 generates a reference image (left side in the drawing) at substantially the same position as the ultrasonic image (right side in the drawing) acquired in real time from the volume data. can be displayed. In addition, by using the Fusion function, the display control function 155f can also display a guideline of the puncture position planned in the puncture plan on the ultrasound image acquired in real time. FIG. 8 is a diagram for explaining an example of the Fusion function according to the first embodiment.

In this way, the positional relationship between the first coordinate space and the second coordinate space is associated by the control function 155a, so that the calculation function 155d can detect the positional deviation between when the puncture is planned and when the puncture is performed. Specifically, the calculation function 155d converts the insertion position (coordinates on the body surface) of the puncture needle planned in the coordinate space (second coordinate space) of the volume data to the position in the first coordinate space. , the deviation from the puncture plan when the puncture needle is inserted at the angle planned in the puncture plan from the insertion position after conversion is calculated.

That is, the calculation function 155d calculates the positional relationship between the predetermined region of the subject and the puncture needle in the puncture plan, and the position of the subject based on the real-time ultrasound image when the puncture needle is inserted in the puncture plan. A difference between the positional relationship between the puncture needle and the predetermined area is calculated. For example, the calculation function 155d calculates the difference between the positional relationship between the target and the puncture needle when the puncture needle is inserted at the insertion position and angle along the puncture plan in the first coordinate space and the planned positional relationship.

The arm control function 155e controls to move the position of the robot arm arranged based on the puncture plan according to the difference in the positional relationship. Specifically, when the first coordinate space and the second coordinate space are associated with each other by the Fusion function, the arm control function 155e establishes a position on the first coordinate space corresponding to the insertion position set in the puncture plan. is identified, and the position of the arm holding the puncture needle is moved so that the puncture needle is inserted from the identified position at the insertion angle set in the puncture plan.

The arm control function 155e controls the position of the subject's body surface acquired from the position sensor 61b, the positions of the arms 22a and 22b acquired from the position sensors 23a and 23b, the puncture needle 31 and the puncture needle. From the shape and size of 32, it is possible to grasp the positional relationship between the subject, the arm, and the puncture needle, so that the arm 22a and the arm 22b can be moved accurately.

Then, the arm control function 155e moves the position of the arm 22c so that the ultrasound probe 12 is placed at a position where the puncture needle held by the arm arranged along the puncture plan is displayed in-plane. Here, the arm control function 155e can move the arm in consideration of other puncture needles, arms, positions of bones, and the like. An in-plane image of the puncture needle can be acquired 360° around the puncture needle, but other puncture needles, arms, and even bone locations limit the range that can be acquired.

Therefore, the arm control function 155e uses the positions of the arms 22a and 22b, the positions of the puncture needles 31 and 32, and the positions of the bones in the subject estimated from the volume data at the time of planning the puncture to determine the position of the arm 22c. determine where to move the For example, the arm control function 155e excludes the position (orientation) where the bone is presumed to enter the image, and from the positions of the arms 22a and 22b and the positions of the puncture needles 31 and 32, Locations where it is difficult to place the ultrasound probe 12 are excluded. Then, the arm control function 155e selects a position from the excluded range and moves the arm 22c.

The arm control function 155e controls the position of the subject's body surface obtained from the position sensor 61b, the position of the arm 22c obtained from the position sensor 23c, and the position of the ultrasonic probe 12 obtained from the position sensor 61a. , and the shape and size of the ultrasonic probe 12, the positional relationship between the subject, the arm, and the ultrasonic probe 12 can be grasped, so that the arm 22c can be moved accurately.

When the arm control function 155e acquires the in-plane image of the puncture needle as described above, the calculation function 155d calculates the positional relationship between the target and the puncture needle when the puncture needle is inserted at the insertion position and angle according to the puncture plan. and the difference from the planned positional relationship. Note that the in-plane image acquired by the ultrasonic probe 12 is an image before the puncture needle is actually inserted, and the puncture needle is not depicted in the ultrasonic image.

For example, the calculation function 155d identifies the position of the puncture needle in the ultrasound image when the puncture needle is inserted at the insertion position and angle according to the puncture plan, and calculates the positional relationship between the identified position and the target. . Here, for example, when the target is included in the in-plane image, the calculation function 155d calculates the positional relationship between the puncture needle and the target in the ultrasound image. On the other hand, if the target is not contained within the in-plane image, calculation function 155d signals arm control function 155e to scan further around the current scan position.

Arm control function 155e, upon receiving a signal from calculation function 155d, controls arm 22c to scan around the current scan position. Here, the plurality of ultrasound images acquired at each position under the control of the arm control function 155e are attached with position information. The calculation function 155d extracts an in-plane ultrasonic image including the target from a plurality of ultrasonic images acquired at each position, and calculates the position of the puncture needle and the position of the target using the extracted ultrasonic image. Calculate relationships.

Here, the calculation function 155d can also use volume data to calculate the positional relationship between the position of the puncture needle and the target. As described above, a plurality of ultrasound images acquired at each position under the control of the arm control function 155e are attached with position information. Therefore, the image generation function 155b generates volume data by synthesizing a plurality of acquired ultrasound images based on position information. The calculation function 155d extracts the positions of the target and the puncture needle in the volume data, and calculates the positional relationship between the position of the puncture needle and the target. As a result, even when there is no ultrasonic image including the target in the in-plane image, the calculation function 155d can calculate the positional relationship between the position of the puncture needle and the target. Note that when synthesizing volume data, it may be updated each time an ultrasonic image is acquired.

The calculation function 155d then compares the calculated positional relationship with the positional relationship between the puncture needle and the target in the puncture plan to calculate the difference. Specifically, the calculation function 155d calculates a difference between the calculated positional relationship and the positional relationship between the puncture needle and the target in the puncture plan, and uses the calculated positional relationship as the positional relationship in the puncture plan. The moving direction and moving amount of the arm are calculated. The calculation function 155d then transmits the calculated movement direction and movement amount to the arm control function 155e. Note that when there is no difference in the positional relationship between the puncture needle and the target in the puncture plan, the movement amount is "0".

The arm control function 155e moves the position of the arm based on the movement direction and movement amount received from the calculation function 155d. As a result, the puncture needle can be positioned accurately even when the position of the organ in the body at the time of puncture is different from the position at the time of planning the puncture.

When the arm 22a and the arm 22b can automatically move the puncture needle, the arm control function 155e moves the position of the arm based on the movement direction and movement amount received from the calculation function 155d. The puncture needle can be inserted into the body by driving the rubber roller 221 so as to move the needle toward the body surface.

Here, the arm control function 155e can control the movement of the arm based on the biological information acquired by the acquisition function 155c. For example, the arm control function 155e detects the body movement of the subject based on biological information, and detects the puncture needle 31 and the puncture needle 32 held by the arms 22a and 22b at a timing when the body movement of the subject is relatively small. control to start the insertion of the For example, the arm control function 155e identifies the timing when the movement of the subject is small based on the respiratory information, ECG, and body surface movement information acquired from the subject, and drives the rubber roller 221 at the identified timing. to control. As a result, it is possible to insert the puncture needle with reduced influence of movement.

Further, for example, the arm control function 155e detects the body movement of the subject based on biological information, and performs control to adjust the positions of the arms 22a and 22b holding the puncture needle according to the body movement of the subject. do. The puncture needle 31 and the puncture needle 32 are held by the arms 22a and 22b until the treatment is completed. Therefore, when the subject moves while the puncture needle is inserted, a load is applied to the site where the puncture needle is inserted. Therefore, the arm control function 155e detects the direction in which the subject moves based on the biological information (for example, information indicating the movement of the body surface) acquired by the acquisition function 155c, and punctures in the same direction as the detected movement. Adjust the position of the arm so that the needle moves.

For example, breathing causes the body surface of the abdomen to reciprocate in the direction in which the puncture needle is inserted. Become. Therefore, the arm control function 155e performs control so as to adjust the positions of the arms 22a and 22b in the insertion direction of the puncture needle according to the movement in the insertion direction of the puncture needle among the body movements of the subject.

In this way, by moving the arm holding the puncture needle according to body movements, the positional relationship between the target and the puncture needle can be kept constant even when the position of the target moves regularly due to respiration. .

The arm control function 155e can control not only the arms 22a and 22b that hold the puncture needles 31 and 32, but also the arm 22c that holds the ultrasonic probe 12 based on biological information. That is, the arm control function 155e can keep the contact state of the ultrasonic probe 12 with the body surface constant by moving the arm 22c following the movement of the body surface.

As described above, the arm control function 155e can automatically adjust the positions of the arms 22a, 22b, and 22c. Furthermore, the arm control function 155e can also move the positions of the arms 22a, 22b, and 22c according to instructions from the operator.

For example, the arm control function 155e moves the positions of the arms 22a, 22b, and 22c according to the operation on the ultrasound image. For example, when the operator designates a position on the ultrasound image, the arm control function 155e controls to move the arm to the designated position.

FIG. 9 is a diagram showing an example of specifying a position via an ultrasound image according to the first embodiment. For example, as shown in FIG. 9, when the operator designates a plurality of puncture needle stop positions on the ultrasound image, the arm control function 155e moves the arm so that the tip of the puncture needle stops at the designated positions. to control. That is, the arm control function 155e acquires the coordinates within the first coordinate space of each puncture needle stop position designated on the ultrasound image, and moves the puncture needle so that the tip of the puncture needle is positioned at the acquired coordinates. to control.

For example, when removing the puncture needle after treating a tumor, the needle is gradually retracted so as not to leave tumor cells in the path through which the needle is pulled out, and cauterization is performed at each position along the path. The puncture needle is removed while performing. In such a case, the puncture needle can be automatically removed simply by designating the position on the ultrasound image.

The images used to specify the position are not only two-dimensional ultrasound images, but also rendered images generated from volume data used for puncture planning, and ultrasound images collected in real time. It may also be possible to use an ultrasound image based on the volume data obtained. Also, the position may be specified not only by arranging a point or line at an arbitrary position, but also by moving the puncture guideline. Furthermore, the position may be specified based on voice input by the operator.

In addition, the collection of in-plane images described above can also be performed according to the operator's designation. For example, in a procedure using a plurality of puncture needles, an in-plane image is observed for each puncture needle in order to confirm that there is no positional deviation after placing the puncture needles.

Therefore, the display control function 155f causes the input interface 14 to display a button for selecting a puncture needle to be visualized in a real-time ultrasonic image. For example, the display control function 155f displays real-time ultrasound images based on the positional relationship between the arm 22c that holds the ultrasound probe 12 that acquires real-time ultrasound images and the arms 22a and 22b that hold the puncture needles. A determination is made as to whether or not the puncture needle can be visualized, and buttons corresponding to the puncture needles that can be visualized are displayed on the input interface 14 .

FIG. 10 is a diagram showing an example of processing by the display control function 155f according to the first embodiment. For example, the display control function 155f controls buttons for displaying in-plane images of the three puncture needles A, B, and C on a touch monitor (touch command screen) that is the input interface 14. to display. The operator can switch between the in-plane images of the puncture needle A, the puncture needle B, and the puncture needle C by touching one of the buttons A to C. That is, the arm control function 155e moves the holding arm so that an in-plane image of the puncture needle corresponding to the touched button is acquired. The display control function 155f causes the display 13 to display the ultrasound image acquired after the movement.

Moreover, when a plurality of puncture needles are used, the plurality of puncture needles may be in-plane in the same image depending on the relative positional relationship of the puncture needles and the performance of the ultrasonic probe. Therefore, the display control function 155f causes the input interface 14 to display a button for simultaneously visualizing a plurality of puncture needles when a plurality of puncture needles can be simultaneously visualized on a real-time ultrasound image. That is, the display control function 155f determines whether or not a plurality of puncture needles are in-plane within the same image. A button is displayed on the touch monitor to display the rendered in-plane image.

FIG. 11 is a diagram showing an example of processing by the display control function 155f according to the first embodiment. For example, when puncture needle B and puncture needle C are in-plane in the same image, the display control function 155f displays buttons (B, C) in addition to buttons A to C, as shown in FIG. Let

The designation of the in-plane image to be collected is performed not only by the touch monitor button described above, but also by inputting identification information for identifying the arm (for example, a number assigned in advance to each arm). good too. Alternatively, a switch may be provided for each arm, and an in-plane image to be acquired may be specified by pressing the switch.

Next, processing of the ultrasonic diagnostic apparatus 100 according to the first embodiment will be described with reference to FIGS. 12 and 13. FIG. 12 and 13 are flowcharts for explaining the procedure of processing of the ultrasonic diagnostic apparatus 100 according to the first embodiment. Here, FIG. 12 shows processing when inserting two puncture needles. Steps S201 and S211 shown in FIG. 12 show procedures performed by the operator before and after processing by the ultrasonic diagnostic apparatus 100 .

Step S202 is a step in which the processing circuit 155 reads a program corresponding to the control function 155a from the memory 154 and executes it. Steps S203-S204, S206-S207, and S209-S210 are steps executed by the processing circuit 155 reading out a program corresponding to the arm control function 155e from the memory 154. FIG. Steps S205 and S208 are steps in which the processing circuit 155 reads out programs corresponding to the calculation function 155d and the arm control function 155e from the memory 154 and executes them.

Steps S<b>301 and S<b>309 shown in FIG. 13 indicate procedures performed by the operator before and after processing by the ultrasonic diagnostic apparatus 100 . Steps S302 to S307 are steps executed by the processing circuit 155 reading programs corresponding to the acquisition function 155c, the calculation function 155d, and the arm control function 155e from the memory 154. FIG. Step S308 is a step in which the processing circuit 155 reads a program corresponding to the arm control function 155e from the memory 154 and executes it.

In the ultrasonic diagnostic apparatus 100 according to the first embodiment, as shown in FIG. 12, when the operator places the patient on the operating table and attaches the position sensor 61b to fix the position (step S201), the processing circuit 155 sets the puncture position based on the puncture plan (step S202). Then, the processing circuit 155 sets the arm 22a holding the puncture needle 31 to the initial position (step S203), and controls the arm 22c so that the puncture needle 31 of the arm 22a is displayed on the ultrasonic diagnostic apparatus. to set the position of the ultrasonic probe 12 (step S204).

Thereafter, processing circuit 155 determines whether or not puncture needle 31 connected to arm 22a is positioned as planned (step S205). Here, if the position is not as planned (No at step S205), the processing circuitry 155 controls the position of the puncture needle 31 based on the real-time ultrasound image (step S206).

In the determination in step S205, if the position is as planned (Yes in step S205), the processing circuit 155 controls the arm 22c so that the puncture needle 32 of the arm 22b is displayed on the ultrasonic diagnostic apparatus. The position of the ultrasonic probe 12 is set (step S207).

Thereafter, processing circuit 155 determines whether or not puncture needle 32 connected to arm 22b is positioned as planned (step S208). Here, if the position is not as planned (No at step S208), the processing circuitry 155 controls the position of the puncture needle 32 based on the real-time ultrasound image (step S209).

In the judgment of step S208, if the position is as planned (Yes in step S208), the processing circuit 155 judges whether or not the puncture operation is completed (step S209). Here, if not completed (No at step S209), the processing circuit 155 returns to step S204 and continues the processing. On the other hand, if completed (Yes at step S209), the operator removes the puncture needle, ultrasonic probe 12, etc. from the patient (step S211).

Further, in the ultrasonic diagnostic apparatus 100 according to the first embodiment, as shown in FIG. 13, when the operator places the patient on the operating table and attaches the position sensor 61b to fix the position (step S301), the process The circuit 155 detects the timing of relatively little body movement based on the biological information (step S202). Although not shown, the processes of steps S202 to S209 shown in FIG. 12 are executed between steps S301 and S302.

Then, the processing circuit 155 starts puncturing at the detected timing and places the puncture needles 31 and 32 (step S303). Then, the processing circuit 155 determines whether or not the puncture needle 31 of the arm 22a is displaced due to body movement (step S304). Here, if there is a deviation (Yes at step S304), the processing circuit 155 controls the arm 22a in accordance with body movement to control the position of the puncture needle 31 (step S305).

After that, the processing circuit 155 determines whether or not the puncture needle 32 of the arm 22b is displaced due to body movement (step S306). Here, if there is a deviation (Yes at step S306), the processing circuit 155 controls the arm 22b in accordance with body movement to control the position of the puncture needle 32 (step S307).

Processing circuitry 155 then determines whether or not the puncture operation is completed (step S308). Here, if not completed (No at step S308), the processing circuit 155 returns to step S304 to continue the processing. On the other hand, if completed (Yes at step S308), the operator removes the puncture needle, ultrasonic probe 12, etc. from the subject (step S309).

As described above, according to the first embodiment, the control function 155a acquires information on the puncture position for the subject that is set based on the puncture plan. The calculation function 155d calculates the difference between the position information of the subject in the puncture position information and the position information of the subject based on real-time ultrasound images acquired from the subject. The arm control function 155e controls to adjust the position of the arm holding the puncture needle that is inserted into the subject based on the puncture plan using the difference. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can adjust the position of the puncture needle based on the actual position at the time of puncture, and can reduce the load associated with puncture.

Further, according to the first embodiment, the calculation function 155d calculates the positional relationship between the predetermined region of the subject in the puncture position information and the puncture needle, and the position of the subject based on the real-time ultrasound image. A difference between the positional relationship between the puncture needle and the predetermined area when the puncture needle is inserted is calculated based on the positional information. The arm control function 155e controls to move the position of the arm arranged based on the puncture plan according to the difference in the positional relationship. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can adjust the positional relationship between the puncture needle and the target in line with the puncture plan, and can easily achieve accurate puncture. do.

Further, according to the first embodiment, the acquisition function 155c acquires biological information of the subject. The arm control function 155e controls to further adjust the arm based on biological information. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment makes it possible to move the arm in accordance with biological information.

Further, according to the first embodiment, the arm control function 155e detects the body movement of the subject based on biological information, and detects the puncture needle held by the arm at a timing when the body movement of the subject is relatively small. control to start the insertion of the Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables highly accurate puncture needle insertion.

Further, according to the first embodiment, the arm control function 155e detects the body movement of the subject based on biological information, and adjusts the position of the arm holding the puncture needle according to the body movement of the subject. to control. Therefore, the ultrasonic diagnostic apparatus 1 according to the first embodiment makes it possible to move the arm according to biological information such as respiration.

Further, according to the first embodiment, the arm control function 155e adjusts the position of the arm in the puncture needle insertion direction in accordance with the body movement of the subject in the puncture needle insertion direction. to control. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment makes it possible to suppress movement of the puncture needle associated with breathing.

Further, according to the first embodiment, the arm control function 155e performs control to adjust the position of the robot arm holding the puncture needle according to the operation on the ultrasound image. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment makes it possible to easily move the arm to a desired position.

Further, according to the first embodiment, the arm control function 155e controls the puncture needle based on the positional relationship between the arm that holds the ultrasound probe that acquires real-time ultrasound images and the arm that holds the puncture needle. The position of the arm holding the ultrasound probe is adjusted so that the is depicted on the real-time ultrasound image. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment makes it possible to increase the degree of freedom in acquiring in-plane images.

Further, according to the first embodiment, the display control function 155f causes the input interface 14 to display a button for selecting a puncture needle to be visualized in a real-time ultrasound image. The display control function 155f determines whether the puncture needle can be visualized in the real-time ultrasound image based on the positional relationship between the arm holding the ultrasound probe 12 that acquires real-time ultrasound images and the arm holding the puncture needle. Then, the input interface 14 is caused to display a button corresponding to the puncture needle that can be drawn. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment makes it possible to easily display an in-plane image.

Further, according to the first embodiment, the display control function 155f provides a button for simultaneously visualizing a plurality of puncture needles in a real-time ultrasound image when the plurality of puncture needles can be simultaneously visualized in the input interface. 14 is displayed. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment makes it possible to easily display an in-plane image including a plurality of puncture needles.

Further, according to the first embodiment, the arm control function 155e controls the positions of the arm holding the puncture needle and the arm holding the ultrasound probe for acquiring real-time ultrasound images based on voice input by the operator. control to adjust Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment makes it possible to easily move the arm.

(Other embodiments)
Now, although the first embodiment has been described so far, it may be implemented in various different forms other than the first embodiment described above.

In the above-described embodiments, the case where the ultrasonic images acquired in real time are two-dimensional images has been described. However, embodiments are not limited to this, and may be the case where three-dimensional ultrasound images are acquired in real time.

Further, in the above-described embodiment, the case where the display control function 155f displays a rendering image generated from volume data or a real-time ultrasound image has been described. However, the display control function 155f can cause various other information to be displayed. For example, the display control function 155f can display an image showing the actual position of the puncture needle on the rendered image generated from the volume data used for the puncture plan.

That is, the display control function 155f controls the puncture needle 31 (puncture needle 32 ). Then, the display control function 155f further identifies a position within the volume data corresponding to the identified position. Thereby, the display control function 155f can display an image showing the puncture needle in the rendered image.

In addition, the display control function 155f controls to present the information of the puncture needle and the image information. For example, the display control function 155f displays a rendered image generated from the volume data used for the puncture plan, and animates the positions of the puncture needle 31, the puncture needle 32, the ultrasonic probe 12, the arm 22a, the arm 22b, and the arm 22c. , and by displaying the scanning position of the ultrasonic probe 12, a live schema function can be realized.

Also, in the above-described embodiment, the case where the position is adjusted using the Fusion function has been described. However, the embodiments are not limited to this, and for example, the Fusion function may not be used. In such cases, for example, segmentation is performed for organs, body surfaces, bones, large vessels, etc. during puncture planning. A target is then specified on an ultrasound image acquired in real time.

The arm control function 155e identifies the positions of organs, bones, and large blood vessels in the first coordinate space by applying segmentation information in a puncture plan to the subject's position information obtained from the position sensor 61b. . Then, the arm control function 155e determines the insertion position and angle of the puncture needle based on the position of the target specified in the real-time ultrasound image and the positions of the internal organs, bones, and large blood vessels in the specified first coordinate space. decide. That is, the arm control function 155e determines a puncture route that avoids bones, blood vessels, and the like. Thereby, the arm control function 155e can automatically adjust the position of the puncture needle without using the Fusion function.

In addition, when the operator designates a target and an insertion position on an ultrasound image, the arm control function 155e controls the position of the arm so that the puncture needle can be inserted from the designated insertion position toward the designated target. can also be controlled.

The arm control function 155e can also control the arm based on the image information of the ultrasound image. For example, in an in-plane image, when the pixel value at the position where the needle exists on the image is not particularly high, the arm control function 155e changes the tilt angle of the ultrasonic probe 12, Control to stop tilting at the angle of luminance.

Further, the display control function 155f can automatically activate an application according to the position of the arm 22c holding the ultrasonic probe 12. FIG. For example, when the arm 22c is positioned to display an in-plane image, applications such as a needle emphasis mode, needle navigation, and puncture line are automatically activated and displayed.

In the above-described embodiment, the case where the puncture assistance device according to the present application is incorporated in the ultrasonic diagnostic apparatus 100 has been described. However, the embodiments are not limited to this, and for example, the puncture assistance device according to the present application may be incorporated in the robot arm device 200. In such a case, the functions described above are performed by processing circuitry 213 .

In addition, the term "processor" used in the above description includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an application specific integrated circuit (ASIC), a programmable logic device ( For example, it means circuits such as Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA). The processor implements its functions by reading and executing programs stored in memory. It should be noted that instead of storing the program in the memory, the program may be directly embedded in the circuit of the processor. In this case, the processor realizes its function by reading and executing the program embedded in the circuit. Note that each processor of the present embodiment is not limited to being configured as a single circuit for each processor, and may be configured as one processor by combining a plurality of independent circuits to realize its function. good.

It should be noted that each component of each device illustrated in the description of the above embodiment is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device can be implemented by a CPU and a program analyzed and executed by the CPU, or implemented as hardware based on wired logic.

Further, the control method described in the above embodiments can be realized by executing a prepared control program on a computer such as a personal computer or a work station. This control program can be distributed via a network such as the Internet. In addition, this control program is recorded on a computer-readable non-temporary recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, DVD, USB memory, and Flash memory such as SD card memory, It can also be executed by being read from a non-transitory recording medium by a computer.

As described above, according to the embodiments, it is possible to reduce the load associated with puncturing.

While several embodiments have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations of embodiments can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention as well as the scope of the invention described in the claims and equivalents thereof.

100 ultrasonic diagnostic apparatus 155, 213 processing circuit 155a control function 155c acquisition function 155d calculation function 155e arm control function 155f display control function

Claims (14)

an acquisition unit that acquires information on the puncture position for the subject that is set based on the puncture plan;
The position information of the subject at the time of the puncture planning in the coordinate space of the ultrasound scan based on the information of the puncture position, and the subject in the coordinate space of the ultrasound scan based on the real-time ultrasound image collected from the subject a calculation unit that calculates the difference from the position information of the sample;
a control unit that controls the position of a robot arm that holds a puncture needle to be inserted into the subject based on the puncture plan so as to be adjusted using the difference;
A puncture assist device. The calculation unit calculates the positional relationship between the predetermined region of the subject and the puncture needle in the puncture position information, and performs the puncture with the puncture position information with respect to the position of the subject based on the real-time ultrasonic image. calculating the difference between the positional relationship between the puncture needle and the predetermined region when the needle is inserted;
The puncture assisting apparatus according to claim 1, wherein the control unit controls to move the position of the robot arm arranged based on the puncture plan according to the difference in the positional relationship. further comprising a biological information acquisition unit that acquires biological information of the subject;
The puncture assistance device according to claim 1 or 2, wherein the control unit controls the robot arm to further adjust the robot arm based on the biological information. The control unit detects the body movement of the subject based on the biological information, and starts inserting the puncture needle held by the robot arm at a timing when the body movement of the subject is relatively small. 4. The puncture assisting device according to claim 3, which controls to 3. The control unit detects a body movement of the subject based on the biological information, and controls the robot arm holding the puncture needle to adjust the position of the robot arm according to the body movement of the subject. 4. The puncture assistance device according to 3. 6. The control unit performs control so as to adjust the position of the robot arm in the puncture direction of the puncture needle in accordance with movement of the puncture needle in the puncture direction of the body movement of the subject. The puncture assist device according to . The puncture assisting apparatus according to any one of claims 1 to 6, wherein the control unit controls to adjust the position of the robot arm holding the puncture needle according to the operation on the ultrasonic image. Based on the positional relationship between a probe robot arm that holds an ultrasonic probe that acquires the real-time ultrasonic image and a robot arm that holds the puncture needle, the control unit controls the position of the puncture needle in the real-time ultrasonic image. The puncture assisting apparatus according to any one of claims 1 to 7, wherein control is performed so as to adjust the position of the probe robot arm so as to be depicted in a sound wave image. further comprising a display control unit for displaying a button for selecting a puncture needle to be depicted in the real-time ultrasound image on the input interface;
The display control unit displays the real-time ultrasound image based on the positional relationship between a probe robot arm that holds an ultrasound probe that acquires the real-time ultrasound image and a robot arm that holds the puncture needle. The puncture assisting device according to claim 8, which determines whether or not a needle can be visualized, and causes the input interface to display a button corresponding to a puncture needle that can be visualized. 10. The display control unit causes the input interface to display a button for simultaneously depicting the plurality of puncture needles when the plurality of puncture needles can be simultaneously depicted in the real-time ultrasound image. The puncture assist device described. The puncture assisting device according to any one of claims 1 to 10, wherein said real-time ultrasound image is a two-dimensional or three-dimensional image. The puncture assisting apparatus according to any one of claims 1 to 11, further comprising a display control unit that controls to present the information of the puncture needle and the image information. The control unit controls to adjust the positions of the robot arm that holds the puncture needle and the probe robot arm that holds the ultrasonic probe that acquires the real-time ultrasonic image, based on voice input by the operator. The puncture assisting device according to any one of claims 1 to 12, wherein an acquisition unit that acquires information on the puncture position for the subject that is set based on the puncture plan;
a calculation unit that calculates the difference between the position information of the subject in the puncture position information and the position information of the subject based on real-time ultrasound images collected from the subject;
a control unit that controls the position of a robot arm that holds a puncture needle to be inserted into the subject based on the puncture plan so as to be adjusted using the difference;
a display control unit for displaying on the input interface a button for selecting a puncture needle to be depicted in the real-time ultrasound image;
with
Based on the positional relationship between a probe robot arm that holds an ultrasonic probe that acquires the real-time ultrasonic image and a robot arm that holds the puncture needle, the control unit controls the position of the puncture needle in the real-time ultrasonic image. Control to adjust the position of the probe robot arm so as to be depicted in the sound wave image,
The display control unit displays the real-time ultrasound image based on the positional relationship between a probe robot arm that holds an ultrasound probe that acquires the real-time ultrasound image and a robot arm that holds the puncture needle. A puncture assisting device that determines whether or not a needle can be visualized, and causes the input interface to display a button corresponding to the puncture needle that can be visualized.

Images