JP2022032595A - Information processing device, information processing system, information processing method and computer program - Google Patents

Abstract

To easily recognize a positional relation between a medical long body and a biological tissue at real time.SOLUTION: A control unit of an information processing device generates a three-dimensional model representing a medical long body on the basis of, a plurality of cross-sectional images at different positions of a biological tissue into which the medical long body is inserted, updates the three-dimensional model on the basis of, contrast images of the biological tissue, and displays, on a display unit, an image representing the medical long body on the basis of, the updated three-dimensional model, along with an image representing the biological tissue.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to information processing devices, information processing systems, information processing methods, and computer programs.

In the treatment of cardiac catheterization for chronic total occlusion (CTO) lesions, it is important to ensure that a medical long body such as a guide wire is passed through the true lumen (lesion area inside the intima of blood vessels). If the guidewire passes through a false lumen outside the true lumen during such a procedure, the vessel wall may dissociate. Good treatment results can be expected by surely grasping the true cavity and performing treatment. In order to ensure that the guide wire passes through the true cavity, it is necessary for the operator to be skilled in the procedure and to accurately grasp the target position (entry point) of the true cavity and the position and posture of the guide wire. Is. "CTO" is an abbreviation for Chronic Total Occlusion.

3D wiring is known as a method of passing a guide wire through a lesion area of a CTO lesion. 3D wiring is a method in which an operator operates a guide wire while estimating the position and posture of the guide wire based on information inside a living tissue such as a blood vessel.

One technique, known as 3D wiring, uses information obtained by taking two contrast images from an orthogonal direction. The surgeon infers the direction of the tip of the guide wire and the relative positional relationship with the entry point from the two contrast images. The operator can recognize the position information of the guide wire according to his / her operation in real time.

Another technique known as 3D wiring uses information obtained by continuously acquiring a cross-sectional image of the tip of a guide wire from a pullback image of an IVUS (intravascular ultrasound) device. The operator observes the cross-sectional image and infers the direction of the tip of the guide wire and the relative positional relationship with the entry point. The surgeon can utilize the pullback image of the IVUS device to recognize the positional relationship between the entry point and the guide wire, and the wire direction. "IVUS" is an abbreviation for Intravascular Ultrasound.

Patent Document 1 describes a technique for displaying an IVUS moving image and an angio moving image (contrast-enhanced moving image).

Japanese Unexamined Patent Publication No. 2010-148778

However, since the contrast image is a two-dimensional image obtained by photographing the object from two directions, it is necessary for the operator to be skilled in recalling the spatial direction of the current guide wire from the two contrast images. I need. In addition, since the lesion is completely stenotic and the contrast medium often does not flow, it is difficult for the operator to confirm the detailed position of the entry point even by referring to the contrast image.

The IVUS appliance can only display one cross-sectional image at a given position in real time. Therefore, in order to observe the change in the direction of the guide wire when the guide wire is operated after the IVUS device pulls back, the operator himself operates the MDU to move the ultrasonic transducer back and forth. You need to shoot again. "MDU" is an abbreviation for Motor Drive Unit.

As described above, in the conventional configuration, it is difficult to recognize the positional relationship between the medical long body such as a guide wire and the entry point in real time, and it is necessary to reliably pass the medical long body through the lesion area. , The skill of the surgeon was required. As a result, in the conventional configuration, the operation takes time and the burden on the patient is large.

An object of the present disclosure is to make it possible to easily recognize the positional relationship between a medical long body and a living tissue in real time.

The information processing apparatus as one aspect of the present disclosure generates a three-dimensional model showing the medical elongated body based on a plurality of cross-sectional images at different positions of the biological tissue into which the medical elongated body is inserted. A control unit that updates the three-dimensional model based on a contrast image of the biological tissue and displays an image showing the medical elongated body on the display unit together with an image showing the biological tissue based on the updated three-dimensional model. To prepare for.

As one embodiment, the control unit updates the three-dimensional model based on the contrast image acquired after at least one of the plurality of cross-sectional images is acquired as the contrast image.

In one embodiment, the control unit analyzes the plurality of cross-sectional images, estimates the position and posture of the medical elongated body in the living tissue, analyzes the contrast image, and estimates the above. A change in at least one of the position and the posture is detected, and the three-dimensional model is updated according to the change.

As one embodiment, the control unit associates the three-dimensional model before the update with a contrast image showing the same position and orientation as the three-dimensional model before the update for the medical long body, and before the update. The change is detected by comparing the contrast image corresponding to the three-dimensional model of the above with the contrast image acquired after the at least one cross-sectional image is acquired.

In one embodiment, the control unit is the control unit after the medical elongated body shown by the three-dimensional model before update is rotated by a predetermined angle about its longitudinal axis. By acquiring a second contrast image of the medical elongated body and comparing the first contrast image, which is the contrast image corresponding to the three-dimensional model before the update, with the second contrast image. The spatial correspondence between the three-dimensional model before the update and the first contrast image is specified.

As one embodiment, the control unit acquires a pair of contrast images taken by two X-ray imaging units as each of the first contrast image and the second contrast image.

As one embodiment, the control unit acquires a pair of radiographic images taken by two X-ray imaging units as a first contrast image which is a contrast image corresponding to the three-dimensional model before update, and said. Based on the position and imaging direction of each of the two X-ray imaging units, the spatial correspondence between the three-dimensional model before updating and the first contrast image is specified.

As one embodiment, the control unit determines a target position of a true cavity in the biological tissue based on the plurality of cross-sectional images, and the display unit includes an image showing the target position in an image showing the biological tissue. To display.

In one embodiment, the control unit is generated as the plurality of cross-sectional images using line data showing the intensity of reflected waves with respect to ultrasonic waves radially transmitted from an ultrasonic vibrator moving inside a blood vessel. , The three-dimensional model is generated based on a plurality of frame images.

The information processing system as one aspect of the present disclosure includes the information processing device, a probe having the ultrasonic vibrator, a photographing device for acquiring the contrast image, and the display unit.

In the information processing method as one aspect of the present disclosure, the control unit of the information processing apparatus shows the medical elongated body based on a plurality of cross-sectional images at different positions of the biological tissue into which the medical elongated body is inserted. A three-dimensional model is generated, the three-dimensional model is updated based on the contrast image of the biological tissue, and an image showing the medical elongated body based on the updated three-dimensional model is displayed as an image showing the biological tissue. Is displayed on the display unit.

The computer program as one aspect of the present disclosure includes a process of generating a three-dimensional model showing the medical elongate based on a plurality of cross-sectional images at different positions of the biological tissue into which the medical elongate is inserted. The process of updating the three-dimensional model based on the contrast image of the living body tissue and the image showing the medical elongated body based on the updated three-dimensional model are displayed on the display unit together with the image showing the living body tissue. Let the computer perform the process of making it.

According to one embodiment of the present disclosure, it is possible to easily recognize the positional relationship between a medical long body and a living tissue in real time.

It is a figure which shows the structure of the diagnosis support system which concerns on one Embodiment of this disclosure. It is a perspective view of the probe and the drive unit which concerns on one Embodiment of this disclosure. It is a perspective view of the X-ray imaging apparatus which concerns on one Embodiment of this disclosure. It is a block diagram which shows the structure of the diagnosis support apparatus which concerns on one Embodiment of this disclosure. It is a figure which shows the example which the IVUS image is acquired by the diagnostic support apparatus which concerns on one Embodiment of this disclosure. It is a figure which shows the example which the IVUS image is acquired by the diagnostic support apparatus which concerns on one Embodiment of this disclosure. It is a figure which shows the example which the IVUS image is acquired by the diagnostic support apparatus which concerns on one Embodiment of this disclosure. It is a figure which shows the example which the IVUS image is acquired by the diagnostic support apparatus which concerns on one Embodiment of this disclosure. It is a figure which shows the example which the IVUS image is acquired by the diagnostic support apparatus which concerns on one Embodiment of this disclosure. It is a flowchart which shows the operation of the diagnosis support apparatus which concerns on one Embodiment of this disclosure. It is a figure explaining the processing example which generates the 3D model based on the cross-sectional image. It is a figure explaining the processing example which specifies the spatial correspondence relationship between a 3D model and a contrast image. It is a figure explaining the processing example which updates a 3D model based on a contrast image. It is a figure explaining the processing example which displays an image based on a 3D model.

Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are designated by the same reference numerals. In the description of the present embodiment, the description will be omitted or simplified as appropriate for the same or corresponding parts.

(System configuration)
With reference to FIG. 1, the configuration of the diagnosis support system 100 as the information processing system according to the present embodiment will be described. FIG. 1 is a diagram showing a configuration of a diagnosis support system 100 according to the present embodiment. The diagnosis support system 100 includes an IVUS device 120, an X-ray imaging device 140, and a diagnosis support device 160 as an information processing device.

The IVUS device 120 is an example of a device that acquires a plurality of cross-sectional images at different positions of a living tissue. In the present embodiment, an ultrasonic image is acquired as such a cross-sectional image, but another imaging device or diagnostic device such as an OCT / OFDI device or an angioscope may be used. "OCT" is an abbreviation for Optical Coherence Tomography. "OFDI" is an abbreviation for Optical Frequency Domain Imaging.

The X-ray imaging apparatus 140 irradiates a living tissue such as a blood vessel into which a contrast medium is charged with X-rays (radiation), detects X-rays transmitted through the living tissue, and detects a contrast image (angio image, X-ray image). , Or also referred to as a radiographic image). As will be described later, the X-ray imaging apparatus 140 includes two sets of X-ray generators and an X-ray camera, and acquires two contrast images taken from different angles.

The diagnosis support device 160 is a dedicated computer specialized for image diagnosis in the present embodiment, but may be a general-purpose computer such as a PC, a WS, or a tablet terminal. "PC" is an abbreviation for Personal Computer. "WS" is an abbreviation for Work Station.

The IVUS device 120, the X-ray imaging device 140, and the diagnostic support device 160 are communicably connected by a cable 12. The IVUS device 120, the X-ray imaging device 140, and the diagnostic support device 160 may be communicably connected by a wireless communication means such as a wireless LAN instead of a wired communication means such as the cable 12. "LAN" is an abbreviation for Local Area Network.

(IVUS device)
The configuration of the IVUS apparatus 120 according to the present embodiment will be described with reference to FIG. FIG. 2 is a perspective view of the probe 20 and the drive unit 13 according to the present embodiment.

The drive unit 13 is used by connecting to the probe 20 and is a device for driving the probe 20. The drive unit 13 is also called an MDU. The probe 20 is applied to IVUS. The probe 20 is also called an IVUS catheter or a diagnostic imaging catheter.

The probe 20 includes a drive shaft 21, a hub 22, a sheath 23, an outer tube 24, an ultrasonic vibrator 25, and a relay connector 26. The drive shaft 21 passes through the sheath 23 inserted into the body cavity of the living body and the outer tube 24 connected to the base end of the sheath 23, and extends to the inside of the hub 22 provided at the base end of the probe 20. The drive shaft 21 has an ultrasonic vibrator 25 for transmitting and receiving signals at its tip, and is rotatably provided in the sheath 23 and the outer tube 24. The relay connector 26 connects the sheath 23 and the outer pipe 24.

The hub 22, the drive shaft 21, and the ultrasonic vibrator 25 are integrally connected to each other so as to move forward and backward in the axial direction of the probe 20. Therefore, for example, when the hub 22 is pushed toward the tip side, the drive shaft 21 and the ultrasonic vibrator 25 move inside the sheath 23 toward the tip side. For example, when the hub 22 is pulled toward the proximal end side, the drive shaft 21 and the ultrasonic transducer 25 move inside the sheath 23 toward the proximal end side as shown by arrows in FIG.

The drive unit 13 includes a scanner unit 31, a slide unit 32, and a bottom cover 33. The scanner unit 31 is connected to the diagnostic support device 160 via the cable 12.

The scanner unit 31 includes a probe connecting portion 34 connected to the probe 20 and a scanner motor 35 which is a drive source for rotating the drive shaft 21. The probe connecting portion 34 is detachably connected to the probe 20 via the insertion port 36 of the hub 22 provided at the base end of the probe 20. Inside the hub 22, the base end of the drive shaft 21 is rotatably supported, and the rotational force of the scanner motor 35 is transmitted to the drive shaft 21. Further, a signal is transmitted and received between the drive shaft 21 and the diagnosis support device 160 via the cable 12. In the diagnosis support device 160, a cross-sectional image of the living lumen is generated and image processing is performed based on the signal transmitted from the drive shaft 21.

The slide unit 32 mounts the scanner unit 31 so as to be able to move forward and backward, and is mechanically and electrically connected to the scanner unit 31. The slide unit 32 includes a probe clamp portion 37, a slide motor 38, and a switch group 39.

The probe clamp portion 37 is provided at a position coaxial with the probe clamp portion 37 on the distal end side of the probe connecting portion 34, and supports the probe 20 connected to the probe connecting portion 34.

The slide motor 38 is a drive source that generates a driving force in the axial direction of the probe 20. The scanner unit 31 moves forward and backward by driving the slide motor 38, and the drive shaft 21 moves forward and backward in the axial direction of the probe 20 accordingly. The slide motor 38 is, for example, a servo motor.

The switch group 39 includes, for example, a forward switch and a pullback switch that are pressed when the scanner unit 31 is moved forward and backward, and a scan switch that is pressed when the image rendering is started and ended. Not limited to the examples given here, various switches are included in the switch group 39 as needed.

When the forward switch is pressed, the slide motor 38 rotates in the forward direction. In response to this, the scanner unit 31 moves forward. As a result, the hub 22 connected to the probe connection portion 34 of the scanner unit 31 is pushed out toward the tip side. The drive shaft 21 and the ultrasonic vibrator 25 move inside the sheath 23 toward the tip side. On the other hand, when the pullback switch is pressed, the slide motor 38 rotates in the reverse direction, and the scanner unit 31 retracts. As a result, the hub 22 connected to the probe connecting portion 34 is swept toward the proximal end side. The drive shaft 21 and the ultrasonic vibrator 25 move inside the sheath 23 toward the proximal end side.

When the scan switch is pressed, image rendering is started, the scanner motor 35 is driven, and the slide motor 38 is driven to retract the scanner unit 31. A user such as an operator connects the probe 20 to the scanner unit 31 in advance so that the drive shaft 21 moves to the axial base end side while rotating around the central axis of the probe 20 at the start of image rendering. do. The scanner motor 35 and the slide motor 38 stop when the scan switch is pressed again, and the image drawing ends.

The bottom cover 33 covers the bottom surface of the slide unit 32 and the entire circumference of the side surface on the bottom surface side, and is freely close to and separated from the bottom surface of the slide unit 32.

(X-ray equipment)
The configuration of the X-ray imaging apparatus 140 according to the present embodiment will be described with reference to FIG. FIG. 3 is a perspective view of the X-ray imaging apparatus 140 according to the present embodiment. The X-ray imaging apparatus 140 includes X-ray generators 51, 53, X-ray cameras 52, 54, arms 55, 56, and an inspection table 57.

The X-ray generation unit 51 irradiates the X-ray camera 52 with X-rays. The X-ray camera 52 detects the X-rays emitted from the X-ray generation unit 51 and generates a contrast image. The arm 55 holds the X-ray generator 51 and the X-ray camera 52. The arm 55 generates X-rays to the subject while maintaining the relative positional relationship between the X-ray generation unit 51 and the X-ray camera 52 so that the X-ray camera 52 can receive the X-rays emitted from the X-ray generation unit 51. The positions and directions of the unit 51 and the X-ray camera 52 can be changed.

The X-ray generation unit 53 irradiates the X-ray camera 54 with X-rays. The X-ray camera 54 detects the X-rays emitted from the X-ray generation unit 53 and generates a contrast image. The arm 56 holds the X-ray generator 53 and the X-ray camera 54. The arm 56 generates X-rays to the subject while maintaining the relative positional relationship between the X-ray generation unit 53 and the X-ray camera 54 so that the X-ray camera 54 can receive the X-rays emitted from the X-ray generation unit 53. The positions and directions of the unit 53 and the X-ray camera 54 can be changed.

The examination table 57 is a table on which the patient who is the subject lies. The X-ray generation unit 51 and the X-ray camera 52, and the X-ray generation unit 53 and the X-ray camera 54, respectively, constitute an X-ray imaging unit for acquiring a contrast image. The contrast image generated by the X-ray camera 52 and the X-ray camera 54 is transmitted to the diagnostic support device 160. In this embodiment, the contrast image is acquired as a moving image.

(Diagnosis support device)
The configuration of the diagnostic support device 160 according to the present embodiment will be described with reference to FIG. FIG. 4 is a block diagram showing the configuration of the diagnostic support device 160 according to the present embodiment. The diagnosis support device 160 includes components such as a control unit 41, a storage unit 42, a communication unit 43, an input unit 44, and an output unit 45.

The control unit 41 is one or more processors. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. "CPU" is an abbreviation for Central Processing Unit. "GPU" is an abbreviation for Graphics Processing Unit. The control unit 41 may include one or more dedicated circuits, or the control unit 41 may replace one or more processors with one or more dedicated circuits. The dedicated circuit is, for example, FPGA or ASIC. "FPGA" is an abbreviation for Field-Programmable Gate Array. "ASIC" is an abbreviation for Application Specific Integrated Circuit. The control unit 41 executes information processing related to the operation of the diagnostic support device 160 while controlling each unit of the diagnostic support system 100 including the diagnostic support device 160.

The storage unit 42 is one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of them. The semiconductor memory is, for example, RAM or ROM. "RAM" is an abbreviation for Random Access Memory (writable memory). "ROM" is an abbreviation for Read Only Memory. The RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for Static Random Access Memory. "DRAM" is an abbreviation for Dynamic Random Access Memory. The ROM is, for example, EEPROM. "EEPROM" is an abbreviation for Electrically Erasable Programmable Read Only Memory. The storage unit 42 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 42 stores information used for the operation of the diagnostic support device 160 and information obtained by the operation of the diagnostic support device 160.

The communication unit 43 is one or more communication interfaces. The communication interface is a wired LAN interface, a wireless LAN interface, or an image diagnostic interface that receives and A / D-converts IVUS signals. "LAN" is an abbreviation for Local Area Network. "A / D" is an abbreviation for Analog to Digital. The communication unit 43 receives the information used for the operation of the diagnostic support device 160, and transmits the information obtained by the operation of the diagnostic support device 160. In the present embodiment, the drive unit 13 is connected to the image diagnosis interface included in the communication unit 43.

The input unit 44 is one or more input interfaces. The input interface is, for example, a USB interface or an HDMI® interface. "HDMI" is an abbreviation for High-Definition Multimedia Interface. The input unit 44 accepts an operation for inputting information used for the operation of the diagnostic support device 160. In the present embodiment, the keyboard and the pointing device are connected to the USB interface included in the input unit 44, but the keyboard and the pointing device may be connected to the wireless LAN interface included in the communication unit 43.

The output unit 45 is one or more output interfaces. The output interface is, for example, a USB interface or an HDMI® interface. The output unit 45 outputs the information obtained by the operation of the diagnosis support device 160. In the present embodiment, a display as a display unit is connected to the HDMI (registered trademark) interface included in the output unit 45.

The function of the diagnosis support device 160 is realized by executing the diagnosis support program (computer program) according to the present embodiment on the processor included in the control unit 41. That is, the function of the diagnostic support device 160 is realized by software. The diagnostic support program is a program for causing a computer to execute a process of a step included in the operation of the diagnostic support device 160, so that the computer can realize a function corresponding to the process of the step. That is, the diagnosis support program is a program for making the computer function as the diagnosis support device 160.

The program can be recorded on a computer-readable recording medium. A computer-readable recording medium is, for example, a magnetic recording device, an optical disk, a photomagnetic recording medium, or a semiconductor memory. The distribution of the program is carried out, for example, by selling, transferring, or renting a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. "DVD" is an abbreviation for Digital Versatile Disc. "CD-ROM" is an abbreviation for Compact Disc Read Only Memory. The program may be distributed by storing the program in the storage of the server and transferring the program from the server to another computer via the network. The program may be provided as a program product.

For example, the computer temporarily stores the program recorded on the portable recording medium or the program transferred from the server in the main storage device. Then, the computer reads the program stored in the main storage device by the processor, and executes the processing according to the read program by the processor. The computer may read the program directly from the portable recording medium and perform processing according to the program. The computer may sequentially execute processing according to the received program each time the program is transferred from the server to the computer. The process may be executed by a so-called ASP type service that realizes the function only by the execution instruction and the result acquisition without transferring the program from the server to the computer. "ASP" is an abbreviation for Application Service Provider. The program includes information used for processing by a computer and equivalent to the program. For example, data that is not a direct command to the computer but has the property of defining the processing of the computer corresponds to "a program-like data".

A part or all the functions of the diagnostic support device 160 may be realized by a dedicated circuit included in the control unit 41. That is, some or all the functions of the diagnostic support device 160 may be realized by hardware. Further, the diagnostic support device 160 may be realized by a single information processing device or may be realized by the cooperation of a plurality of information processing devices.

(Acquisition of IVUS image)
The principle of acquiring an IVUS image will be described with reference to FIGS. 5A-5E. 5A-5E are diagrams showing an example in which an IVUS image is acquired by the diagnostic support device 11 according to the present embodiment.

As described above, during the acquisition of the IVUS image, the drive shaft 21 is pulled back to the proximal end side where the drive unit 13 exists in accordance with the retreat of the scanner unit 31 while rotating around the central axis of the probe 20. The ultrasonic vibrator 25 provided at the tip of the drive shaft 21 rotates around the central axis of the probe 20 in conjunction with the drive shaft 21 while transmitting and receiving ultrasonic signals inside a living tissue such as a blood vessel. While moving to the proximal end side of the probe 20.

FIG. 5A schematically shows a plane orthogonal to the central axis of the probe 20 passing through the position where the ultrasonic transducer 25 is present. The ultrasonic transducer 25 radially transmits ultrasonic pulse waves while rotating at a constant velocity around the central axis of the probe 20 in the living tissue, and receives reflected waves of pulse waves from each direction. To generate a received signal. 5A-5E show an example in which 512 pulse waves are transmitted at equal intervals while the ultrasonic vibrator 25 rotates 360 degrees, and a reception signal of reflected waves from each direction is generated. In this way, each direction in which the ultrasonic pulse is transmitted is called a line (scanning line) around the entire circumference of the ultrasonic transducer 25, and data showing the intensity of the reflected wave of the ultrasonic wave in each line is called line data.

FIG. 5B shows an example of acquiring a luminance image by performing luminance conversion of the received signal of the reflected wave in each line. As shown in FIG. 5B, the received signal of each line is converted into a detection waveform indicating its amplitude, and logarithmic conversion is performed. Then, the luminance conversion is performed in order to associate the signal intensity with the luminance in the image. The control unit 41 of the diagnostic support device 11 allocates 256 levels of grayscale colors according to the intensity of the reflected signal, and constructs luminance data for one line. The horizontal axis of each figure in FIG. 5B indicates time. The vertical axis of the received signal, the detection waveform, and the logarithmic conversion in FIG. 5B shows the signal strength. The speed at which ultrasonic waves propagate is constant. Therefore, the time from transmitting the ultrasonic wave to receiving the reflected wave is proportional to the distance between the ultrasonic vibrator 25 and the object reflecting the ultrasonic wave. Therefore, the line data obtained by the process of FIG. 5B shows the intensity of the reflected signal according to the distance from the ultrasonic transducer 25 by the brightness of the image.

FIG. 5C is an example in which the line data of each line 1 to 512 that has undergone luminance conversion are stacked and displayed. By transforming this image into polar coordinates, a cross-sectional image of a living tissue as shown in FIG. 5D is acquired. One cross-sectional image is generated from 512 line data acquired while the ultrasonic transducer 25 rotates 360 degrees around the central axis of the probe 20. Such a single cross-sectional image is called a frame image, and the data is called frame data. The position of each pixel included in the frame image related to the IVUS image is specified by the distance r from the ultrasonic vibrator 25 and the rotation angle θ from the reference angle of the ultrasonic vibrator 25.

As described above, the acquisition of the IVUS image is performed while the ultrasonic transducer 25 is pulled back to the proximal end side of the probe 20. Therefore, as shown in FIG. 5E, when the ultrasonic transducer 25 moves in the blood vessel, the imaging catheter (probe 20) is swept (pulled back) in the long axis direction of the blood vessel to acquire a continuous frame image 61. It will be. Here, the ultrasonic transducer 25 acquires frame data one by one while moving. Therefore, the long axis direction of the blood vessel in the plurality of frame data acquired continuously corresponds to the time direction. Hereinafter, the two orthogonal directions in the same frame image are referred to as spatial directions to distinguish them from the time direction. In FIG. 5E, the x-direction and the y-direction correspond to the spatial direction. The z direction corresponds to the time direction. In FIGS. 5A to 5E, an example in which the number of line data acquired for each rotation of the probe 20 is 512 has been described, but the number of line data is not 512, for example, 256 or 1024. It may be an individual or the like.

(Operation example)
The control unit 41 of the diagnostic support device 160 as an information processing device according to the present embodiment is based on a plurality of cross-sectional images of a biological tissue into which a medical long body (guide wire, catheter, etc.) is inserted, based on medical treatment. Generate a three-dimensional model showing a long body. Then, the control unit 41 updates the three-dimensional model based on the contrast image of the living tissue, and displays an image showing the medical long body based on the updated three-dimensional model on the display unit together with the image showing the living tissue. Let me. Therefore, according to the configuration according to the present embodiment, this positional relationship can be easily recognized by displaying the positional relationship between the medical long body and the living tissue in real time.

The operation of the diagnostic support system 100 according to the present embodiment will be described with reference to FIG. The operation of the diagnostic support device 160 described with reference to FIG. 6 corresponds to the information processing method according to the present embodiment. The operation of each step in FIG. 6 is executed based on the control of the control unit 41. In this embodiment, an example of using an IVUS image as a cross-sectional image will be described, but an image acquired by another imaging device such as an OCT / OFDI device or a blood vessel endoscope may be used.

Prior to the start of the flow of FIG. 6, the probe 20 is fitted into the probe connection portion 34 and the probe clamp portion 37 of the drive unit 13 and connected and fixed to the drive unit 13. Then, the probe 20 is inserted to the target site in the blood vessel. The diagnostic support system 100 pulls back the ultrasonic transducer 25 while rotating it around the central axis of the probe 20, and starts taking an IVUS image. Further, the diagnostic support system 100 pulls back the ultrasonic transducer 25 while rotating it around the central axis of the probe 20, and starts taking a contrast image of the same affected area in parallel with taking an IVUS image. do.

In step S11, the control unit 41 generates a three-dimensional model based on the IVUS image as a cross-sectional image. FIG. 7 illustrates an example of generating a 3D model based on an IVUS image.

In FIG. 7, 71 is a cross-sectional view of a blood vessel as a living tissue. 65 is the true lumen of the blood vessel. 66 is a false lumen of a blood vessel. At the boundary between the true cavity 65 and the false cavity 66, the endothelium 68 of the blood vessel is present. A sheath 23 attached to the guide wire 63 is inserted into the blood vessel 71. In the sheath 23, the drive shaft 21 and the ultrasonic vibrator 25 are integrally connected to each other so as to move forward and backward in the axial direction of the sheath 23. Yet another guide wire 64 is inserted into the blood vessel 71.

The ultrasonic vibrator 25 transmits ultrasonic waves radially while moving in the sheath 23 by pullback, receives the reflected wave, and notifies the diagnostic support device 160 of the intensity of the reflected wave. The control unit 41 of the diagnosis support device 160 acquires a continuous frame image 61 based on the intensity of the notified reflected wave. Within the living tissue, each region such as the blood flow region, the metal, the true cavity, and the false lumen has different ultrasonic reflection characteristics. Therefore, the control unit 41 analyzes each of the continuous frame images 61 to identify the guide wire 64 and the true cavity 65 (lower left in FIG. 7). The control unit 41 generates a three-dimensional model 72 which is three-dimensional data showing the true cavity 65 and the guide wire 64 of the blood vessel from the continuous frame image 61 in which the guide wire 64 and the true cavity 65 are identified. At that time, the control unit 41 analyzes a plurality of cross-sectional images. As a result, the control unit 41 estimates the position and posture of the medical long body in the living tissue. In this way, the control unit 41 generates the three-dimensional model 72. The control unit 41 determines a target position (entry point) of the true cavity in the living tissue based on a plurality of cross-sectional images. For example, the control unit 41 sets the location where the width of the true cavity 65 is the narrowest as the entry point 67 in the three-dimensional model 72.

In step S12, the control unit 41 identifies the spatial correspondence between the three-dimensional model and the contrast image. Then, the control unit 41 associates the three-dimensional model before the update with the contrast image showing the same position and posture as the three-dimensional model before the update for the medical long body. FIG. 8 illustrates an example of processing for specifying the spatial correspondence between the three-dimensional model and the contrast image.

In the example of FIG. 8, the control unit 41 stores the three-dimensional model 72 of the guide wire 64 and the contrast images 74 and 75 generated by the cross-sectional pixels obtained by the IVUS device 120. After that, the control unit 41 instructs the surgeon to operate the guide wire 64 in a predetermined direction and angle (for example, 45 ° clockwise), and the surgeon operates the guide wire 64. The pair of contrast-enhanced images obtained as a result of the operation is stored. By performing this operation a plurality of times, the 3D model 72 of the guide wire 64 and the contrast images 74 and 75 can be associated with each other without the position information of the X-ray imaging unit of the X-ray imaging apparatus 140. In this way, the control unit 41 is the medical length after the medical long body shown by the three-dimensional model before the update is rotated by a predetermined angle about the axis in the longitudinal direction thereof. A second contrast image of the scale is acquired. Then, the control unit 41 compares the first contrast image, which is the contrast image corresponding to the three-dimensional model before the update, with the second contrast image, so that the three-dimensional model before the update and the first contrast are contrasted. Identify the spatial correspondence with the image. According to such a method, it is possible to correspond the 3D model 72 of the guide wire 64 with the contrast images 74 and 75 without the position information of the X-ray imaging apparatus 140. As each of the first contrast image and the second contrast image, a pair of contrast images taken by the two X-ray imaging units can be acquired. In this way, the control unit 41 identifies the spatial correspondence between the 3D model and the contrast image, and the positions of the 3D model before the update and the medical elongated body are the same as those of the 3D model before the update. And the contrast image showing the posture.

The method for specifying the spatial correspondence between the three-dimensional model and the contrast image is not limited to the above. For example, the control unit 41 acquires the position information of the two X-ray imaging units of the X-ray imaging apparatus 140 in advance, and based on the positions and imaging directions of the two X-ray imaging units, before the update. The spatial correspondence between the three-dimensional model and the pair of contrast images may be specified. This makes it possible to specify the spatial correspondence between the three-dimensional model and the contrast-enhanced image without performing the above-mentioned calibration. However, if the guide wire 64 looks linear in any one of the pair of contrast images, a unique solution cannot be obtained. Therefore, if the guide wire 64 looks linear in any one of the pair of contrast images, the control unit 41 may instruct the operator to rotate the guide wire 64 a little. Specifically, the control unit 41 may display to that effect on the display 92 connected to the output unit 45, or may give an instruction by voice. Then, the control unit 41 may specify the spatial correspondence after the rotation operation.

Further, in the present embodiment, the X-ray imaging apparatus 140 includes two X-ray imaging units, but may include only one X-ray imaging unit. In such a case, the X-ray generator and the arm provided with the X-ray camera rotate around the patient, and the 3D model and the guide wire are based on the change in the shape of the guide wire 64 taken at that time. The spatial correspondence with 64 may be specified.

In step S13, the control unit 41 updates the three-dimensional model based on the contrast image. FIG. 9 illustrates an example of a process of updating a three-dimensional model based on a contrast image.

In FIG. 9, 81 is a three-dimensional model before updating. 82 and 83 are a pair of contrast images associated with the 3D model 81. The time t when the three-dimensional model 81 is acquired is 0.

After that, it is assumed that the operator operates the guide wire 64 and the contrast images 84 and 85 are acquired at time t = t ₁ . In this case, the control unit 41 is a three-dimensional model corresponding to the shape of the guide wire 64 shown in the contrast images 84 and 85 based on the spatial correspondence between the three-dimensional model specified in step 12 and the contrast image. The 86 is acquired, and the 3D model 81 is updated by the 3D model 86. That is, the control unit 41 detects at least one change in the estimated position and posture of the conventional guide wire 64 by analyzing the contrast images 84 and 85. Then, the control unit 41 updates the three-dimensional model according to the change.

In step S14, the control unit 41 displays an image based on the three-dimensional model updated in step S13. FIG. 10 illustrates an example of a process of displaying an image based on a three-dimensional model.

In FIG. 10, 92 is a display as a display unit connected to the diagnostic support device 160. The display 92 displays an IVUS image captured by the IVUS apparatus 120, a contrast image captured by the X-ray imaging apparatus 140, and an image 91 corresponding to the three-dimensional model. In the image 91, based on the three-dimensional model, an image showing the guide wire 64 and an image of the living tissue including the entry point 67 are displayed so that the spatial positional relationship between them can be understood. Therefore, the surgeon can easily recognize the positional relationship between the medical long body and the living tissue in real time by referring to the image 91. In the image 91 of FIG. 10, the guide wire 64 is arranged in the foreground and the entry point 67 is arranged in the back, and the appearance of observing these from diagonally above is shown. However, the guide wire 64 and the entry point 67 in the image 91 are shown. The arrangement of is not limited to this. For example, the longitudinal axis of the guide wire 64 may be arranged so as to extend laterally in the image 91. Further, the arrangement of the guide wire 64 and the entry point 67 displayed on the image 91 may be changed according to the operation of the keyboard or the like connected to the input unit 44. This allows the operator to confirm the positional relationship between the guide wire 64 and the entry point 67 from a direction suitable for performing the procedure.

In step S15, the control unit 41 determines whether or not to end the process. When it is determined to end the process (YES in step S15), the control unit 41 ends the process. If it is not determined to end the process (NO in step S15), the control unit 41 returns to step S13 and continues the process.

As described above, the control unit 41 of the diagnostic support device 160 generates a three-dimensional model showing the medical long body based on a plurality of cross-sectional images at different positions of the biological tissue into which the medical long body is inserted. .. Then, the control unit 41 updates the three-dimensional model based on the contrast image of the living tissue, and displays an image showing the medical long body based on the updated three-dimensional model on the display unit together with the image showing the living tissue. Let me. Therefore, according to the configuration of the present embodiment, the image of the three-dimensional model showing the medical elongated body corresponding to the update of the contrast image is displayed in real time together with the image showing the living tissue, so that the surgeon can perform medical treatment. The positional relationship between the long body and the living tissue can be easily recognized.

Further, the control unit 41 updates the three-dimensional model based on the contrast image acquired after at least one of the plurality of cross-sectional images is acquired as the contrast image. Therefore, according to the configuration of the present embodiment, it is possible to display an image of a three-dimensional model corresponding to the latest contrast image.

Further, the control unit 41 associates the three-dimensional model before the update with the contrast image showing the same position and posture as the three-dimensional model before the update for the medical long body. After that, the control unit 41 compares the contrast-enhanced image corresponding to the three-dimensional model before the update with the contrast-enhanced image acquired after at least one cross-sectional image is acquired, so that the long medical body is estimated. Detects changes in at least one of the position and posture of. Therefore, according to the present embodiment, it is possible to display an image showing the positional relationship between the medical long body and the biological tissue updated according to the change in the position and posture of the medical long body.

In the above embodiment, an example of generating a three-dimensional model using an IVUS image as a cross-sectional image has been described. That is, based on a plurality of frame images generated as a plurality of cross-sectional images using line data showing the intensity of the reflected wave with respect to the ultrasonic waves radially transmitted from the ultrasonic vibrator 25 moving inside the blood vessel. An example of generating a 3D model was explained. However, as the cross-sectional image, an image acquired by, for example, an OCT / OFDI device, a blood vessel endoscope, or the like may be used instead of the IVUS image.

The present disclosure is not limited to the embodiments described above. For example, the plurality of blocks described in the block diagram may be integrated. Alternatively, one block may be divided into a plurality of blocks. The steps described in the flow chart may be performed in parallel or in a different order, depending on the processing power of the device performing each step, or as needed, instead of being performed in chronological order as described. good. Other changes are possible without departing from the spirit of this disclosure.

12 Cable 13 Drive unit 20 Probe 21 Drive shaft 22 Hub 23 Sheath 24 Outer tube 25 Ultrasonic transducer 26 Relay connector 31 Scanner unit 32 Slide unit 33 Bottom cover 34 Probe connection 35 Scanner motor 36 Outlet 37 Probe clamp 38 Slide motor 39 Switch group 41 Control unit 42 Storage unit 43 Communication unit 44 Input unit 45 Output unit 51 X-ray generator 52 X-ray camera 53 X-ray generator 54 X-ray camera 55 Arm 56 Arm 57 Inspection table 61 Frame image 63 Guide Wire 64 Guide wire 65 True cavity 66 False cavity 67 Entry point 68 Endoline 72 3D model 74 Contrast image 75 Contrast image 81 3D model 82 Contrast image 83 Contrast image 84 Contrast image 85 Contrast image 86 3D model 91 Image 92 Display 100 Diagnostic support system 120 IVUS device 140 X-ray equipment 160 Diagnostic support device

Claims (12)

Based on a plurality of cross-sectional images at different positions of the biological tissue into which the medical elongate was inserted, a three-dimensional model showing the medical elongate was generated.
The three-dimensional model is updated based on the contrast image of the living tissue.
An information processing device including a control unit that displays an image showing the medical long body on the display unit together with an image showing the living tissue based on the updated three-dimensional model. The control unit
The information processing apparatus according to claim 1, wherein the three-dimensional model is updated based on the contrast image acquired after at least one of the plurality of cross-sectional images is acquired as the contrast image. The control unit
By analyzing the plurality of cross-sectional images, the position and posture of the medical long body in the living tissue are estimated.
The information processing apparatus according to claim 2, wherein the contrast image is analyzed to detect a change in at least one of the estimated positions and postures, and the three-dimensional model is updated according to the change. The control unit
The three-dimensional model before the update is associated with a contrast image showing the same position and posture as the three-dimensional model before the update for the medical long body.
The information processing according to claim 3, wherein the change is detected by comparing the contrast image corresponding to the three-dimensional model before the update with the contrast image acquired after the at least one cross-sectional image is acquired. Device. The control unit
A second aspect of the medical elongate after the medical elongate shown by the three-dimensional model before the update has been rotated by a predetermined angle about its longitudinal axis. Get a contrast image,
By comparing the first contrast image, which is a contrast image corresponding to the three-dimensional model before the update, with the second contrast image, the three-dimensional model before the update and the first contrast image can be compared. The information processing apparatus according to claim 4, which specifies a spatial correspondence. The control unit
The information processing apparatus according to claim 5, wherein a pair of contrast images taken by two X-ray photographing units are acquired as each of the first contrast image and the second contrast image. The control unit
As the first contrast image which is the contrast image corresponding to the three-dimensional model before the update, a pair of contrast images taken by the two X-ray imaging units was acquired.
The information processing according to claim 4, which specifies the spatial correspondence between the three-dimensional model before update and the first contrast image based on the position and the imaging direction of each of the two X-ray imaging units. Device. The control unit
Based on the plurality of cross-sectional images, the target position of the true cavity in the living tissue is determined.
The information processing apparatus according to any one of claims 1 to 7, wherein an image showing the living tissue includes an image showing the target position and displayed on the display unit. The control unit
3 The information processing apparatus according to any one of claims 1 to 8, which generates a three-dimensional model. The information processing apparatus according to claim 9 and
With the probe having the ultrasonic vibrator,
An imaging device that acquires the contrast image and
An information processing system including the display unit. The control unit of the information processing device
Based on a plurality of cross-sectional images at different positions of the biological tissue into which the medical elongate was inserted, a three-dimensional model showing the medical elongate was generated.
The three-dimensional model is updated based on the contrast image of the living tissue.
An information processing method for displaying an image showing the medical long body on the display unit together with an image showing the living tissue based on the updated three-dimensional model. A process of generating a three-dimensional model showing the medical elongate based on a plurality of cross-sectional images at different positions of the biological tissue into which the medical elongate is inserted.
The process of updating the three-dimensional model based on the contrast-enhanced image of the living tissue, and
A computer program that causes a computer to perform a process of displaying an image showing a medical long body on a display unit together with an image showing a living tissue based on the updated three-dimensional model.

Images