JP7480009B2 - Information processing device, information processing system, information processing method, and computer program - Google Patents

Description

The present disclosure relates to an information processing device, an information processing system, an information processing method, and a computer program.

A configuration is known in which IVUS images taken by an IVUS (intravascular ultrasound) device and contrast images taken by an X-ray device are displayed to assist surgeons such as doctors in their procedures. "IVUS" is an abbreviation for Intravascular Ultrasound.

Patent document 1 describes technology related to displaying ultrasound images.

JP 2017-140135 A

IVUS images are acquired as a group of tomographic images taken continuously mainly at the tip of a guide wire inserted into the body during the pullback operation of the IVUS device. Contrast images are acquired as planar images obtained by irradiating X-rays from outside the body and detecting the X-rays that pass through the body. Thus, IVUS images and contrast images differ in the direction of shooting or the nature of the images obtained by shooting, and there are generally no common coordinates for both. Therefore, an operator using a configuration that displays IVUS images and contrast images had to match the position and direction in the IVUS image with the position and direction in the contrast image in his own brain. For example, in order to recognize which direction the direction of the lesion interpreted on the IVUS image corresponds to on the contrast image, the operator had to determine the direction of the branching blood vessel from the IVUS image of the blood vessel long axis in his own brain and compare it with the cine image. Such interpretation work is complicated and requires the operator's experience to perform it.

The purpose of this disclosure is to make it easy to recognize what a certain direction is in an IVUS image.

An information processing device according to one aspect of the present disclosure is an information processing device communicatively connected to an ultrasound transducer that transmits a first ultrasound wave from a body surface toward inside the body by an ultrasound transmitter and transmits a second ultrasound wave radially and receives a reflected wave of the second ultrasound wave inside living tissue, and includes a control unit that acquires line data from the ultrasound transducer that indicates the ultrasound wave received by the ultrasound transducer, generates a frame image that represents at least the inside of the living tissue based on the intensity of the reflected wave indicated by the line data, displays the generated frame image on a display unit, and, when the first ultrasound wave is indicated by the line data, displays the frame image on the display unit together with an image indicating the direction in which the first ultrasound wave was received by the ultrasound transducer.

In one embodiment, the control unit separates the first ultrasonic wave and the reflected wave from the line data at different times in the same direction by independent component analysis, and determines whether the ultrasonic wave indicated by the line data corresponds to the reflected wave or the first ultrasonic wave.

In one embodiment, the control unit determines the direction in which the ultrasonic transducer received the first ultrasonic wave based on the direction of line data that indicates the strongest ultrasonic wave among the line data acquired by the ultrasonic transducer.

In one embodiment, the control unit rotates the frame image and displays it on the display unit so that the direction in which the ultrasound transducer receives the first ultrasound coincides with a predetermined reference direction.

An information processing system according to one aspect of the present disclosure includes the information processing device, the ultrasonic transmitter, and the ultrasonic transducer.

In one embodiment, the system further includes an imaging device that acquires a contrast image of the biological tissue, the information processing device is communicably connected to the imaging device, the control unit causes the display unit to display the contrast image acquired by the imaging device together with the frame image, and the reference direction is determined according to the imaging direction of the imaging device.

An information processing method according to one aspect of the present disclosure is an information processing method of an information processing device communicatively connected to an ultrasound transducer that transmits a first ultrasound wave from a body surface toward inside the body by an ultrasound transmitter and transmits a second ultrasound wave radially and receives a reflected wave of the second ultrasound wave inside living tissue, in which a control unit acquires line data from the ultrasound transducer indicating the ultrasound wave received by the ultrasound transducer, generates a frame image representing at least the inside of the living tissue based on the intensity of the reflected wave indicated by the line data, displays the generated frame image on a display unit, and, when the first ultrasound wave is indicated by the line data, displays the frame image on the display unit together with an image indicating the direction in which the first ultrasound wave was received by the ultrasound transducer.

An information processing method according to one aspect of the present disclosure is an information processing method for an information processing system including an ultrasound transmitter that transmits a first ultrasound from a body surface toward inside the body, an ultrasound transducer that receives the first ultrasound transmitted by the ultrasound transmitter inside the biological tissue and transmits a second ultrasound radially to receive a reflected wave of the second ultrasound, and an information processing device communicatively connected to the ultrasound transducer, in which a control unit of the information processing device acquires line data from the ultrasound transducer that indicates the ultrasound received by the ultrasound transducer, generates a frame image that represents at least the inside of the biological tissue based on the intensity of the reflected wave indicated by the line data, displays the generated frame image on a display unit, and, when the first ultrasound is indicated by the line data, displays the frame image on the display unit together with an image indicating the direction in which the ultrasound transducer received the first ultrasound.

A computer program as one aspect of the present disclosure is a computer program for a computer communicatively connected to an ultrasound transducer that receives inside living tissue a first ultrasound wave emitted from an ultrasound transmitter from a body surface toward inside the body and a second ultrasound wave that is transmitted radially and receives a reflected wave of the second ultrasound wave, and causes the computer to execute the following processes: acquiring line data from the ultrasound transducer that indicates the ultrasound wave received by the ultrasound transducer; generating a frame image that represents at least the inside of the living tissue based on the intensity of the reflected wave indicated by the line data; displaying the generated frame image on a display unit; and, when the first ultrasound wave is indicated by the line data, displaying the frame image on the display unit together with an image indicating the direction in which the ultrasound transducer received the first ultrasound wave.

The present disclosure makes it possible to easily recognize what direction a certain direction is in an IVUS image, thereby facilitating the task of interpreting IVUS images and contrast images in association with each other.

1 is a diagram illustrating a configuration of a diagnosis support system according to an embodiment of the present disclosure. FIG. 2 is a perspective view of a probe and a drive unit according to an embodiment of the present disclosure; FIG. 1 is a perspective view of an X-ray imaging apparatus according to an embodiment of the present disclosure. 1 is a block diagram showing a configuration of a diagnosis support device according to an embodiment of the present disclosure. 1 is a diagram showing an example in which an IVUS image is acquired by a diagnosis support device according to an embodiment of the present disclosure. 1 is a diagram showing an example in which an IVUS image is acquired by a diagnosis support device according to an embodiment of the present disclosure. 1 is a diagram showing an example in which an IVUS image is acquired by a diagnosis support device according to an embodiment of the present disclosure. 1 is a diagram showing an example in which an IVUS image is acquired by a diagnosis support device according to an embodiment of the present disclosure. 1 is a diagram showing an example in which an IVUS image is acquired by a diagnosis support device according to an embodiment of the present disclosure. 1A and 1B are diagrams illustrating an example of imaging by a diagnosis support system according to an embodiment of the present disclosure. FIG. 13 is a diagram showing an example of an IVUS image displayed together with an image showing the reception direction of ultrasonic waves from an ultrasonic transmitter. FIG. 13 is a diagram showing an example of an IVUS image displayed after being rotated so that the reception direction of ultrasonic waves from an ultrasonic transmitter coincides with a predetermined reference direction. FIG. 11 is a diagram showing an example of a screen on which an IVUS image and a contrast image are displayed in a diagnosis support system according to an embodiment of the present disclosure. 5 is a flowchart illustrating an operation of the diagnosis support device according to an embodiment of the present disclosure. FIG. 13 is a schematic diagram of an example of acquiring data at different times on the same line. FIG. 13 is a schematic diagram showing examples of IVUS images when the body surface wave is ON and OFF. FIG. 13 is a schematic diagram showing a process for determining the direction of a body surface wave. FIG. 13 is a schematic diagram showing a process for determining the direction of a body surface wave.

Below, one embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are given the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

(System configuration)
The configuration of a diagnosis support system 100 as an information processing system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of the diagnosis support system 100 according to this embodiment. The diagnosis support system 100 includes an IVUS device 120, an X-ray imaging device 140 as an imaging device that obtains contrast images of biological tissues, a diagnosis support device 160 as an information processing device, and an ultrasonic transmitter 180 as an ultrasonic transmitter.

The X-ray imaging device 140 is a device that irradiates biological tissue, such as blood vessels into which a contrast agent has been injected, with X-rays (radiation) and detects the X-rays that have passed through the biological tissue to obtain a contrast image. As described below, the X-ray imaging device 140 is equipped with two sets of X-ray generators and an X-ray camera, and obtains two contrast images taken from different angles. However, the X-ray imaging device 140 may be configured to include one set of X-ray generators and an X-ray camera. A contrast image is also called an angio image, an X-ray image, or a radiological image. A contrast image of a moving image may also be called a cine image.

In this embodiment, the diagnostic support device 160 is a dedicated computer specialized for image diagnosis, but it may be a general-purpose computer such as a PC, WS, or tablet terminal. "PC" is an abbreviation for Personal Computer. "WS" is an abbreviation for Work Station.

The IVUS device 120, the X-ray imaging device 140, and the diagnostic support device 160 are communicatively connected by a cable 12. The IVUS device 120, the X-ray imaging device 140, and the diagnostic support device 160 may be communicatively connected by a wireless communication means such as a wireless LAN, instead of a wired communication means such as the cable 12. "LAN" is an abbreviation for Local Area Network.

(IVUS device)
The configuration of the IVUS device 120 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a perspective view of the probe 20 and the drive unit 13 according to this embodiment.

The drive unit 13 is a device that is connected to the probe 20 and drives the probe 20. The drive unit 13 is also called an MDU. The probe 20 is applied to IVUS. The probe 20 is also called an IVUS catheter or an imaging diagnostic catheter.

The probe 20 comprises a drive shaft 21, a hub 22, a sheath 23, an outer tube 24, an ultrasonic transducer 25, and a relay connector 26. The drive shaft 21 passes through the sheath 23, which is inserted into the body cavity of the living body, and the outer tube 24, which is connected to the base end of the sheath 23, and extends to the inside of the hub 22, which is provided at the base end of the probe 20. The drive shaft 21 has an ultrasonic transducer 25 at its tip, which transmits and receives signals, and is provided rotatably within the sheath 23 and the outer tube 24. The relay connector 26 connects the sheath 23 and the outer tube 24.

The hub 22, the drive shaft 21, and the ultrasonic transducer 25 are connected to each other so that they move forward and backward in the axial direction of the probe 20 as a unit. Therefore, for example, when the hub 22 is pushed toward the tip side, the drive shaft 21 and the ultrasonic transducer 25 move toward the tip side inside the sheath 23. For example, when the hub 22 is pulled toward the base end side, the drive shaft 21 and the ultrasonic transducer 25 move toward the base end side inside the sheath 23 as shown by the arrow in FIG. 2. As described later, the ultrasonic transducer 25 transmits and receives ultrasonic signals inside biological tissue such as blood vessels, while rotating around the central axis of the probe 20 in conjunction with the drive shaft 21, and moves inside the probe 20 to obtain a tomographic image. In this embodiment, the ultrasonic transducer 25 also receives ultrasonic waves transmitted from the body surface toward the inside of the body by the ultrasonic transmitter 180.

The drive unit 13 includes a scanner unit 31, a slide unit 32, and a bottom cover 33. The scanner unit 31 is connected to the diagnostic support device 160 via a cable 12.

The scanner unit 31 includes a probe connection section 34 that connects to the probe 20, and a scanner motor 35 that is a drive source that rotates the drive shaft 21. The probe connection section 34 is detachably connected to the probe 20 via a socket 36 of a hub 22 provided at the base end of the probe 20. Inside the hub 22, the base end of the drive shaft 21 is rotatably supported, and the rotational force of the scanner motor 35 is transmitted to the drive shaft 21. In addition, signals are transmitted and received between the drive shaft 21 and the diagnostic support device 160 via the cable 12. In the diagnostic support device 160, cross-sectional images of the biological lumen are generated and image processing is performed based on the signal transmitted from the drive shaft 21.

The slide unit 32 carries the scanner unit 31 so that it can move back and forth, and is mechanically and electrically connected to the scanner unit 31. The slide unit 32 includes a probe clamp section 37, a slide motor 38, and a group of switches 39.

The probe clamp section 37 is located coaxially with and closer to the tip of the probe connection section 34, and supports the probe 20 that is connected to the probe connection section 34.

The slide motor 38 is a drive source that generates a driving force in the axial direction of the probe 20. The scanner unit 31 moves forward and backward when driven by the slide motor 38, and the drive shaft 21 moves forward and backward in the axial direction of the probe 20 accordingly. The slide motor 38 is, for example, a servo motor.

The switch group 39 includes, for example, a forward switch and a pullback switch that are pressed when moving the scanner unit 31 forward and backward, and a scan switch that is pressed when starting and ending image drawing. The switches are not limited to the examples given here, and various other switches may be included in the switch group 39 as necessary.

When the forward switch is pressed, the slide motor 38 rotates forward. In response, the scanner unit 31 moves forward. This causes the hub 22 connected to the probe connection portion 34 of the scanner unit 31 to be pushed toward the tip side. The drive shaft 21 and the ultrasonic transducer 25 move toward the tip side inside the sheath 23. On the other hand, when the pullback switch is pressed, the slide motor 38 rotates in the reverse direction, and the scanner unit 31 moves backward. This causes the hub 22 connected to the probe connection portion 34 to be swept toward the base end. The drive shaft 21 and the ultrasonic transducer 25 move toward the base end inside the sheath 23.

When the scan switch is pressed, image drawing begins, and the scanner motor 35 and slide motor 38 are driven to move the scanner unit 31 backward. A user such as an operator connects the probe 20 to the scanner unit 31 in advance, so that when image drawing begins, the drive shaft 21 moves axially toward the base end while rotating around the central axis of the probe 20. When the scan switch is pressed again, the scanner motor 35 and slide motor 38 stop, and image drawing ends.

The bottom cover 33 covers the bottom surface of the slide unit 32 and the entire side surface on the bottom side, and can be moved toward and away from the bottom surface of the slide unit 32.

(X-ray imaging device)
The configuration of the X-ray imaging apparatus 140 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a perspective view of the X-ray imaging apparatus 140 according to this embodiment. The X-ray imaging apparatus 140 includes X-ray generation units 52 and 54, X-ray cameras 51 and 53, arms 55 and 56, and an examination table 57.

The X-ray generating unit 52 irradiates X-rays toward the X-ray camera 51. The X-ray camera 51 detects the X-rays irradiated from the X-ray generating unit 52 and generates a contrast image. The arm 55 holds the X-ray generating unit 52 and the X-ray camera 51. The arm 55 can change the position and direction of the X-ray generating unit 52 and the X-ray camera 51 relative to the subject while maintaining the relative positional relationship between the X-ray generating unit 52 and the X-ray camera 51 so that the X-rays irradiated from the X-ray generating unit 52 can be received by the X-ray camera 51.

The X-ray generating unit 54 irradiates X-rays toward the X-ray camera 53. The X-ray camera 53 detects the X-rays irradiated from the X-ray generating unit 54 and generates a contrast image. The arm 56 holds the X-ray generating unit 54 and the X-ray camera 53. The arm 56 can change the position and direction of the X-ray generating unit 54 and the X-ray camera 53 relative to the subject while maintaining the relative positional relationship between the X-ray generating unit 54 and the X-ray camera 53 so that the X-rays irradiated from the X-ray generating unit 54 can be received by the X-ray camera 53.

The examination table 57 is a table on which the patient, who is the subject, lies. The X-ray generating unit 52 and the X-ray camera 51, and the X-ray generating unit 54 and the X-ray camera 53, respectively constitute an X-ray imaging unit that acquires contrast images. The contrast images generated by the X-ray camera 51 and the X-ray camera 53 are transmitted to the diagnosis support device 160. In this embodiment, the contrast images are acquired as moving images (cine images).

(Diagnosis Support Device)
The configuration of the diagnosis support device 160 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the diagnosis support device 160 according to this embodiment. The diagnosis support device 160 includes components such as a control unit 41, a storage unit 42, a communication unit 43, an input unit 44, and an output unit 45.

The control unit 41 is one or more processors. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. "CPU" is an abbreviation for Central Processing Unit. "GPU" is an abbreviation for Graphics Processing Unit. The control unit 41 may include one or more dedicated circuits, or one or more processors in the control unit 41 may be replaced with one or more dedicated circuits. The dedicated circuit is, for example, an FPGA or an ASIC. "FPGA" is an abbreviation for Field-Programmable Gate Array. "ASIC" is an abbreviation for Application Specific Integrated Circuit. The control unit 41 controls each part of the diagnosis support system 100 including the diagnosis support device 160, and executes information processing related to the operation of the diagnosis support device 160.

The storage unit 42 is one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. The semiconductor memory is, for example, a RAM or a ROM. "RAM" is an abbreviation for Random Access Memory. "ROM" is an abbreviation for Read Only Memory. The RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for Static Random Access Memory. "DRAM" is an abbreviation for Dynamic Random Access Memory. The ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for Electrically Erasable Programmable Read Only Memory. The storage unit 42 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 42 stores information used in the operation of the diagnosis support device 160 and information obtained by the operation of the diagnosis support device 160.

The communication unit 43 is one or more communication interfaces. The communication interface is a wired LAN interface, a wireless LAN interface, or an image diagnosis interface that receives and A/D converts IVUS signals. "LAN" is an abbreviation for Local Area Network. "A/D" is an abbreviation for Analog to Digital. The communication unit 43 receives information used in the operation of the diagnostic support device 160, and transmits information obtained by the operation of the diagnostic support device 160. In this embodiment, the drive unit 13 is connected to the image diagnosis interface included in the communication unit 43.

The input unit 44 is one or more input interfaces. The input interface is, for example, a USB interface or an HDMI (registered trademark) interface. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface. The input unit 44 accepts an operation to input information used in the operation of the diagnostic support device 160. In this embodiment, a keyboard and a pointing device are connected to a USB interface included in the input unit 44, but a keyboard and a pointing device may also be connected to a wireless LAN interface included in the communication unit 43.

The output unit 45 is one or more output interfaces. The output interface is, for example, a USB interface or an HDMI (registered trademark) interface. The output unit 45 outputs information obtained by the operation of the diagnostic support device 160. In this embodiment, a display serving as a display unit is connected to the HDMI (registered trademark) interface included in the output unit 45.

The functions of the diagnostic assistance device 160 are realized by executing a diagnostic assistance program (computer program) according to this embodiment on a processor included in the control unit 41. That is, the functions of the diagnostic assistance device 160 are realized by software. The diagnostic assistance program is a program for causing a computer to execute processing of steps included in the operation of the diagnostic assistance device 160, thereby causing the computer to realize functions corresponding to the processing of those steps. That is, the diagnostic assistance program is a program for causing a computer to function as the diagnostic assistance device 160.

The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic recording devices, optical disks, magneto-optical recording media, and semiconductor memories. The program may be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs or CD-ROMs on which the program is recorded. "DVD" is an abbreviation for Digital Versatile Disc. "CD-ROM" is an abbreviation for Compact Disc Read Only Memory. The program may be distributed by storing the program in the storage of a server and transferring the program from the server to other computers via a network. The program may be provided as a program product.

The computer temporarily stores in the main storage device, for example, a program recorded on a portable recording medium or a program transferred from a server. The computer then reads the program stored in the main storage device with a processor and executes processing according to the read program with the processor. The computer may read the program directly from the portable recording medium and execute processing according to the program. The computer may execute processing according to the received program each time a program is transferred to the computer from the server. Processing may be executed by a so-called ASP-type service that does not transfer a program from the server to the computer and achieves functions only by issuing execution instructions and obtaining results. "ASP" is an abbreviation for Application Service Provider. Programs include information used for processing by a computer and equivalent to a program. For example, data that is not a direct command to a computer but has properties that define computer processing falls under " equivalent to a program".

Some or all of the functions of the diagnostic support device 160 may be realized by a dedicated circuit included in the control unit 41. In other words, some or all of the functions of the diagnostic support device 160 may be realized by hardware. In addition, the diagnostic support device 160 may be realized by a single information processing device, or may be realized by the cooperation of multiple information processing devices.

(Ultrasonic transmitter)
The ultrasonic transmitter 180 transmits ultrasonic waves from the body surface to the inside of the body, which can be received by the ultrasonic transducer 25 inserted inside the living tissue. The ultrasonic transmitter 180 has a mechanism for adjusting at least one of the amplitude, frequency, and phase of the transmitted ultrasonic waves. The ultrasonic transmitter 180 transmits ultrasonic waves having at least one of the amplitude, frequency, and phase different from the ultrasonic waves transmitted by the ultrasonic transducer 25. This allows the diagnosis support device 11 to determine whether the ultrasonic waves indicated by the line data are reflected waves of ultrasonic waves transmitted from the ultrasonic transducer 25 or ultrasonic waves transmitted from the ultrasonic transmitter 180, focusing on at least one of the amplitude, frequency, and phase. In addition, as described later, the ultrasonic transmitter 180 is used to identify the direction serving as a reference in the IVUS image, unlike the case of being used in a normal echo examination. Therefore, the ultrasonic transmitter 180 has a mechanism for attaching the ultrasonic transmitter 180 to the body surface, so that the ultrasonic transmitter 180 can be fixed to the body surface and the direction serving as a reference in the IVUS image can be prevented from moving during the treatment.

(Acquisition of IVUS images)
The principle of acquiring an IVUS image will be described with reference to Figures 5A to 5E, which are diagrams showing an example of acquiring an IVUS image by the diagnosis support device 11 according to this embodiment.

As described above, during IVUS imaging, the drive shaft 21 rotates around the central axis of the probe 20 and is pulled back to the base end where the drive unit 13 is located as the scanner unit 31 retracts. The ultrasound transducer 25 provided at the tip of the drive shaft 21 moves towards the base end of the probe 20 while rotating around the central axis of the probe 20 in conjunction with the drive shaft 21, while transmitting and receiving ultrasound signals inside biological tissue such as blood vessels.

Figure 5A shows a schematic diagram of a plane perpendicular to the central axis of the probe 20, passing through the position where the ultrasonic transducer 25 is located. The ultrasonic transducer 25 transmits ultrasonic pulse waves radially while rotating at a uniform speed around the central axis of the probe 20 in the biological tissue, receives reflected waves of the pulse waves from each direction, and generates a received signal. Figures 5A to 5E show an example in which the ultrasonic transducer 25 transmits 512 pulse waves at equal intervals while rotating 360 degrees, and generates received signals of the reflected waves from each direction. In this way, each direction in which ultrasonic pulses are transmitted around the entire circumference of the ultrasonic transducer 25 is called a line (scanning line), and data indicating the reflected waves of ultrasonic waves in each line is called line data.

Figure 5B shows an example of obtaining a luminance image by performing luminance conversion on the received signal of the reflected wave on each line. As shown in Figure 5B, the received signal on each line is converted into a detection waveform indicating its amplitude, and then logarithmic conversion is performed. Then, luminance conversion is performed to associate the signal strength with the luminance in the image. The control unit 41 of the diagnosis support device 11 constructs luminance data for one line by, for example, assigning 256 grayscale colors according to the strength of the reflected signal. The horizontal axis of each diagram in Figure 5B indicates time. The vertical axis of the received signal, detection waveform, and logarithmic conversion in Figure 5B indicates signal strength. The speed at which ultrasound travels is constant. Therefore, the time from transmitting ultrasound to receiving the reflected wave is proportional to the distance between the ultrasound transducer 25 and the object that reflected the ultrasound. Therefore, the line data obtained by the processing in Figure 5B indicates the strength of the reflected signal according to the distance from the ultrasound transducer 25 by the luminance of the image.

Figure 5C shows an example in which the line data of lines 1 to 512 that have been subjected to brightness conversion are stacked and displayed. By converting this image into polar coordinates, a cross-sectional image of biological tissue as shown in Figure 5D is obtained. One cross-sectional image is generated from 512 pieces of line data acquired while the ultrasound transducer 25 rotates 360 degrees around the central axis of the probe 20. Such a cross-sectional image is called a frame image, and the data is called frame data. An image generated from a frame image is called an IVUS image. The position of each pixel included in the frame image related to the IVUS image is specified by the distance r from the ultrasound transducer 25 and the rotation angle θ from the reference angle of the ultrasound transducer 25.

As described above, the IVUS image is acquired while the ultrasound transducer 25 is pulled back toward the base end of the probe 20. Therefore, as shown in FIG. 5E, when the ultrasound transducer 25 moves in the blood vessel, the imaging catheter (probe 20) is swept (pulled back) in the blood vessel long axis direction to acquire successive frame images 60. Here, the ultrasound transducer 25 acquires frame data one by one while moving. Therefore, the long axis direction of the blood vessel in the consecutively acquired multiple frame data corresponds to the time direction. Hereinafter, the two orthogonal directions in the same frame image are referred to as spatial directions to distinguish them from the time direction. In FIG. 5E, the x direction and the y direction correspond to the spatial direction. The z direction corresponds to the time direction. Note that, in FIG. 5A to FIG. 5E, an example in which the number of line data acquired per one rotation of the probe 20 is 512 has been described, but the number of line data may not be 512, but may be, for example, 256 or 1024.

(Example of operation)
The diagnostic support device 160 is communicably connected to an ultrasonic transducer 25 that transmits a first ultrasonic wave from the surface of the body toward the inside of the body by an ultrasonic transmitter 180 and transmits a second ultrasonic wave radially and receives a reflected wave of the second ultrasonic wave inside the living tissue. The control unit 41 of the diagnostic support device 160 acquires line data indicating the ultrasonic wave received by the ultrasonic transducer 25 from the ultrasonic transducer 25. The control unit 41 generates a frame image representing at least the inside of the living tissue based on the intensity of the reflected wave indicated by the line data. Then, the control unit 41 displays the generated frame image on a display as a display unit. When the first ultrasonic wave transmitted from the ultrasonic transmitter 180 is indicated by the line data, the control unit 41 displays the frame image on the display together with an image indicating the direction in which the ultrasonic transducer 25 received the first ultrasonic wave. Therefore, according to the configuration of this embodiment, it is possible to grasp an IVUS image in association with the direction of the ultrasonic wave transmitted by the ultrasonic transmitter 180, and it becomes possible to easily recognize the shooting direction of the IVUS image.

The operation of the diagnostic support system 100 according to this embodiment will be described with reference to Figures 6 to 9.

Figure 6 is a diagram for explaining an example of imaging by the diagnosis support system 100. In Figure 6, 200 is the body of a patient who is the subject. In Figure 6, the body 200 is a schematic diagram of a patient lying on his back on an examination table 57, with a cross section of the chest viewed from the head. 210 is the patient's heart. 220 is a blood vessel present in the heart 210. In Figure 6, the probe 20 is inserted into the blood vessel 220 of the heart 210, and the ultrasonic transducer 25 transmits and receives ultrasonic waves within the blood vessel 220. At the same time, the blood vessel 220 is imaged by the X-ray generating unit 52 and the X-ray camera 51. Furthermore, an ultrasonic transmitter 180 is attached to the surface of the body 200. The ultrasonic transmitter 180 can be provided near the area where ultrasonic waves pass between the X-ray generating unit 52 and the X-ray camera 51. As a result, the diagnostic support device 160 displays an image indicating the direction of the ultrasound transmitter 180 along with the frame image, allowing the user to easily understand the positional relationship between the IVUS image and the contrast image acquired by the X-ray generator 52 and X-ray camera 51.

In this configuration, the ultrasonic transducer 25 receives not only the reflected waves of the ultrasonic waves transmitted from the ultrasonic transducer 25, but also the ultrasonic waves transmitted from the ultrasonic transmitter 180, and transmits line data indicating these ultrasonic waves to the diagnostic support device 160. The control unit 41 generates a frame image based on the reflected waves of the ultrasonic waves transmitted from the ultrasonic transducer 25, identifies the ultrasonic waves transmitted from the ultrasonic transmitter 180, and causes the display unit to display an image indicating the direction from which the ultrasonic transducer 25 received the ultrasonic waves together with the frame image.

7A is a diagram showing an example of an IVUS image displayed together with an image showing the reception direction of the ultrasound from the ultrasound transmitter 180. In FIG. 7A, 61 is a frame image generated based on the reflected waves of the ultrasound transmitted from the ultrasound transducer 25. 62 is an image showing the reception direction of the ultrasound from the ultrasound transmitter 180. 65 corresponds to the direction in which the body surface exists. 63 indicates the part where the lesion is found. 66 indicates the direction in which the lesion exists. In this way, by also showing the reception direction of the ultrasound transmitted from the ultrasound transmitter 180 in the IVUS image, the surgeon can grasp the IVUS image and can easily recognize the shooting direction of the IVUS image. In particular, when the ultrasound transmitter 180 is provided near the area where the ultrasound passes between the X-ray generating unit 52 and the X-ray camera 51, the surgeon can easily grasp the correspondence between the IVUS image and the contrast image.

7B is a diagram showing an example of an IVUS image in which an image 62 indicating the reception direction and a frame image 61 are rotated so that the reception direction of the ultrasound from the ultrasound transmitter 180 coincides with a predetermined reference direction. In the example of FIG. 7B, the reference direction is provided at the top of the screen, and an image 62 indicating the reception direction of the ultrasound from the ultrasound transmitter 180 is attached to the top of the frame image 61. In this way, by rotating and displaying the IVUS image so that the reception direction of the ultrasound from the ultrasound transmitter 180 coincides with the predetermined reference direction, the operator can more easily recognize the shooting direction of the IVUS image. This reference direction can be determined according to the shooting direction of the X-ray imaging device 140. This allows the operator to easily recognize the correspondence between the contrast image captured by the X-ray imaging device 140 and the IVUS image.

In addition, instead of such a display, the control unit 41 may rotate and display the image 62 indicating the receiving direction and the frame image 61 so that the image 62 indicating the receiving direction is displayed in a direction corresponding to the spatial direction in which the ultrasonic transmitter 180 emits ultrasonic waves toward the patient. For example, the control unit 41 may rotate and display the image 62 indicating the receiving direction and the frame image 61 so that the image 62 indicating the receiving direction is located at a position rotated from the top of the screen by an angle between the direction of the ultrasonic waves emitted by the ultrasonic transmitter 180 and the vertical downward direction. The direction of the ultrasonic waves emitted by the ultrasonic transmitter 180 is acquired by measuring the attitude of the ultrasonic transmitter 180 using a gyro sensor or the like. In this way, the control unit 41 rotates and displays the image 62 indicating the receiving direction and the frame image 61 so that the image 62 indicating the receiving direction is located at a position corresponding to the attitude of the ultrasonic transmitter 180, making it possible for the operator to more easily recognize the shooting direction of the IVUS image.

Figure 8 is a diagram showing an example of a screen on which an IVUS image and a contrast image are displayed in the diagnosis support system 100. 81 is a screen displayed on the display. A frame image 61 acquired by the IVUS device 120 and a contrast image 71 acquired by the X-ray imaging device 140 are displayed on the screen 81. The frame image 61 is displayed together with an image 62 showing the reception direction of the ultrasound from the ultrasound transmitter 180. As described with reference to Figures 7A and 7B, the direction of the lesion corresponds to the traveling direction of the ultrasound from the ultrasound transmitter 180. In the contrast image 71, the ellipse pointed to by the arrow 72 indicates the direction of the lesion. Therefore, the surgeon who observes the screen 81 can grasp at a glance the relationship between the direction in the frame image 61 and the direction in the contrast image 71, and can easily interpret the contrast image.

Figure 9 is a flowchart showing the operation of the diagnosis support device 160. The operation of the diagnosis support device 160 described with reference to Figure 9 corresponds to the information processing method according to this embodiment. The operation of each step in Figure 9 is executed based on the control of the control unit 41.

Before the flow of FIG. 6 begins, the user fits the probe 20 into the probe connection portion 34 and the probe clamp portion 37 of the drive unit 13, and connects and fixes it to the drive unit 13. The probe 20 is then inserted to the target site in the blood vessel. The diagnostic support system 100 starts capturing IVUS images by pulling back the ultrasound transducer 25 while rotating it around the central axis of the probe 20. The diagnostic support system 100 also starts capturing contrast images of the same affected area in parallel with capturing IVUS images by pulling back the ultrasound transducer 25 while rotating it around the central axis of the probe 20.

In step S11, the control unit 41 acquires line data indicating ultrasonic waves received by the ultrasonic transducer 25 rotating inside the probe 20. Such ultrasonic waves are reflected waves in response to signal waves transmitted radially from the ultrasonic transducer 25 toward the outside of the probe 20 at predetermined time intervals, or ultrasonic waves transmitted from the ultrasonic transmitter 180 (hereinafter referred to as "body surface waves").

In step S12, the control unit 41 determines whether the ultrasound indicated by the line data acquired in step S11 is a reflected wave of the ultrasound transmitted from the ultrasound transducer 25. If it is a reflected wave (YES in step S12), the control unit 41 performs the processes in steps S13 and onward. If the ultrasound indicated by the line data is not a reflected wave, i.e., if it is a body surface wave (NO in step S12), the control unit 41 performs the processes in steps S15 and onward.

The control unit 41 can determine whether the ultrasound indicated by the line data corresponds to a reflected wave or a body surface wave based on at least one of the amplitude, phase, and frequency of the ultrasound indicated by the line data. Specifically, the control unit 41 can use an independent component analysis (ICA) technique to distinguish between reflected waves and body surface waves from a signal wave in which both are mixed. ICA is an abbreviation for Independent Component Analysis. The control unit 41 may separate reflected waves and body surface waves from data at different times on the same line using an independent component analysis technique. FIG. 10 is a schematic diagram of an example of acquiring data at different times on the same line. For example, the control unit 41 acquires line data a92, a'93, ... at different times on the same line from a frame image 91 in which reflected waves and body surface waves are mixed. The control unit 41 may use an independent component analysis technique to distinguish between the two, assuming that the line data is data generated by mixing reflected waves and body surface waves, which are independent data.

Alternatively, the ultrasonic transmitter 180 may periodically repeat transmission ("ON") and non-transmission ("OFF") of pulsed body surface waves at a predetermined timing. FIG. 11 is a schematic diagram showing an example of an IVUS image when the body surface waves are ON and OFF. The control unit 41 may distinguish between reflected waves and body surface waves based on the periodic change between a frame image 94 when the body surface waves are ON and a frame image 95 when the body surface waves are OFF. In other words, by repeatedly turning the body surface waves ON and OFF, the reflected waves and body surface waves are visually highlighted. Therefore, the control unit 41 may separate the reflected waves and body surface waves from a frame image in which multiple input data are thought to exist by data processing.

Furthermore, the control unit 41 may distinguish between the reflected waves and the body surface waves based on the depths reached by the two. In a frame image, it is common that an image based on the body surface waves from the ultrasonic transmitter 180 can be observed at a greater distance than the ultrasonic transducer 25, compared to an image based on the reflected waves of the ultrasonic waves radially transmitted from the ultrasonic transducer 25. Therefore, the control unit 41 may determine the body surface waves based on data on the depths at which the reflected waves do not reach on the frame image, i.e., the reflected waves do not return and cannot be observed on the frame image, but the body surface waves can be observed.

Using these techniques, the control unit 41 can determine whether the ultrasound indicated by the line data is a reflected wave of the ultrasound transmitted from the ultrasound transducer 25.

In step S13, the control unit 41 generates a frame image using the line data acquired in step S11. The specific processing procedure for generating a frame image is as described above. Note that, if sufficient line data to generate one frame image has not been received, the processing of steps S13 and S14 is skipped and the process returns to step S11.

In step S14, the control unit 41 displays the frame image generated in step S13 on a display serving as a display unit. The control unit 41 then performs the processes from step S19 onward.

In step S15, the control unit 41 determines the direction in which the ultrasonic transducer 25 receives the body surface waves transmitted from the ultrasonic transmitter 180. Specifically, the control unit 41 determines the direction in which the strength of the ultrasonic waves received from the body surface is the highest as the receiving direction of the body surface waves. Figures 12A and 12B are schematic diagrams showing the process of determining the direction of the body surface waves. As shown in Figure 12A, the ultrasonic transmitter 180 transmits ultrasonic waves from the body surface of the human body 200. In parallel with this, the ultrasonic transducer 25 in the blood vessel 220 transmits and receives ultrasonic waves. In such a case, the IVUS image formed by the ultrasonic transducer 25 is as shown in Figure 12B. In Figure 12B, the frame image 96 shows the reception of ultrasonic waves with strong strength in the direction 97 in which ultrasonic waves are received from the ultrasonic transducer 25, based on line data 98 by the body surface waves. Therefore, the control unit 41 determines the direction in which the ultrasonic transducer 25 received the body surface waves transmitted from the ultrasonic transmitter 180 based on the direction of the line data that indicates the strongest ultrasonic waves among the line data acquired by the ultrasonic transducer 25.

In step S16, the control unit 41 determines whether or not to rotate the frame image based on the direction in which the ultrasound transducer 25 receives the body surface waves. This determination can be made, for example, by having the user set whether or not to rotate the frame image prior to processing and referring to the setting. Note that it is also possible to allow the user to set whether or not to rotate the frame image during processing. If it is determined that the frame image is to be rotated (YES in step S16), the control unit 41 performs processing from step S17 onwards. If it is not determined that the frame image is to be rotated (NO in step S16), the control unit 41 performs processing from step S18 onwards.

In step S17, the control unit 41 rotates the frame image and the image indicating the direction of reception by the ultrasonic transducer 25 according to the direction of reception by the ultrasonic transducer 25. For example, the control unit 41 may rotate the frame image so that the direction in which the ultrasonic transducer 25 receives the ultrasonic waves from the ultrasonic transmitter 180 matches a predetermined reference direction. Alternatively, the control unit 41 may rotate the frame image according to the attitude of the ultrasonic transmitter 180, for example.

In step S18, the control unit 41 displays an image indicating the direction in which the ultrasonic transducer 25 received the ultrasonic waves from the ultrasonic transmitter 180 on the display as a display unit. If the frame image is rotated in step S17, the control unit 41 displays an image indicating the direction in which the ultrasonic transducer 25 received the ultrasonic waves from the ultrasonic transmitter 180 rotated accordingly. The processing of steps S13 and S14 and the processing of steps S15 to S18 can be executed in parallel. Therefore, if the series of processing of steps S13 to S18 shows that ultrasonic waves from the ultrasonic transmitter 180 are indicated by the line data, the frame image can be displayed on the display together with an image indicating the direction in which the ultrasonic transducer 25 received the ultrasonic waves.

In step S19, the control unit 41 acquires the contrast image captured by the X-ray imaging device 140 from the X-ray imaging device 140.

In step S20, the control unit 41 displays the contrast image acquired in step S19 on a display as a display unit. As described above, the control unit 41 displays a frame image together with an image showing the reception direction of the ultrasound from the ultrasound transmitter 180 (steps S14 and S18), and displays the contrast image in step S19. Therefore, the surgeon can easily recognize the position and direction in the IVUS image and the position and direction in the contrast image in association with each other. In particular, the control unit 41 may rotate the frame image and display it on the display unit so that the direction in which the ultrasound transducer 25 receives the ultrasound from the ultrasound transmitter 180 coincides with the reference direction determined according to the imaging direction of the X-ray imaging device 140. In this case, the surgeon can intuitively and easily recognize the position and direction in the IVUS image and the position and direction in the contrast image in association with each other.

In step S21, the control unit 41 determines whether or not to end the process. If the process is to be ended (YES in step S20), the control unit 41 ends the process. If the process is not to be ended (NO in step S20), the control unit 41 returns to step S11 and continues the process.

As described above, in the diagnostic support system 100, the ultrasound transducer 25 receives not only the reflected waves of the ultrasound transmitted by the ultrasound transducer 25, but also ultrasound from the ultrasound transmitter 180 provided on the body surface, and obtains the transmission direction of the ultrasound on the frame image. This associates the direction on the frame image with the body surface direction, making it possible to utilize the relationship between the direction on the body surface and the direction in the contrast image. This makes it easier to read the direction of the lesion on the contrast image.

By using the diagnostic support system 100 in intravascular treatment, the surgeon can easily check the relationship between the captured image and the IVUS image when advancing the wire into the biological tissue. Therefore, the surgeon can easily check the branching of blood vessels and proceed with the treatment, which makes it possible to shorten the procedure time. In addition, for example, when performing directional coronary angioplasty (DCA), it is possible to improve the accuracy of determining the direction of DCA and reduce the risk of vascular perforation. "DCA" is an abbreviation for directional coronary atherectomy.

For ease of explanation, FIG. 9 shows an example in which the CPU 41 performs processing to generate frame images in steps S13 and S14, and processing to display an image indicating the receiving direction of the body surface waves in steps S15 to S18. For ease of explanation, FIG. 9 also shows an example in which the CPU 41 displays a contrast image after performing processing in steps S11 to S18. However, these processes do not need to be performed in this order, and may be executed in parallel, for example.

The present disclosure is not limited to the above-described embodiments. For example, multiple blocks shown in the block diagram may be integrated. Alternatively, one block may be divided. Multiple steps shown in the flowchart may be executed in parallel or in a different order depending on the processing capacity of the device executing each step or as needed, instead of being executed chronologically as described. Other modifications are possible without departing from the spirit of the present disclosure.

REFERENCE SIGNS LIST 12 cable 13 drive unit 20 probe 21 drive shaft 22 hub 23 sheath 24 outer tube 25 ultrasonic transducer 26 relay connector 31 scanner unit 32 slide unit 33 bottom cover 34 probe connection section 35 scanner motor 36 socket 37 probe clamp section 38 slide motor 39 switch group 41 control section 42 memory section 43 communication section 44 input section 45 output section 51 X-ray generation section 52 X-ray camera 53 X-ray generation section 54 X-ray camera 55 arm 56 arm 57 examination table 60 frame image 100 diagnosis support system 120 IVUS device 140 X-ray imaging device 160 diagnosis support device 180 ultrasonic transmitter 200 human body

Claims (9)

An information processing device is communicably connected to an ultrasonic transducer that receives a first ultrasonic wave emitted from an ultrasonic transmitter from a body surface toward inside the body and a second ultrasonic wave radially transmitted and a reflected wave of the second ultrasonic wave inside a living tissue,
acquiring line data from the ultrasonic transducer, the line data indicating the ultrasonic waves received by the ultrasonic transducer;
generating a frame image representing at least an inside of the biological tissue based on the intensity of the reflected wave represented by the line data;
Displaying the generated frame image on a display unit;
a control unit that, when the first ultrasonic wave is indicated by the line data, causes a display unit to display the frame image together with an image indicating a direction in which the first ultrasonic wave is received by the ultrasonic transducer ;
the ultrasonic transmitter has an attitude measuring unit that measures an attitude of the ultrasonic transmitter,
The control unit is
determining a reference direction according to the attitude of the ultrasonic transmitter measured by the attitude measurement unit;
rotate the frame image and display it on the display unit so that a direction in which the ultrasonic transducer receives the first ultrasonic wave coincides with the reference direction;
Information processing device. The information processing device according to claim 1, wherein the control unit separates the first ultrasonic wave and the reflected wave from the line data at different times in the same direction by independent component analysis, and determines whether the ultrasonic wave indicated by the line data corresponds to the reflected wave or the first ultrasonic wave. The information processing device according to claim 1 or 2, wherein the control unit determines the direction in which the ultrasonic transducer receives the first ultrasonic wave based on the direction of line data that indicates the strongest ultrasonic wave among the line data acquired by the ultrasonic transducer. 4. The information processing device according to claim 1, wherein the control unit determines, as the reference direction, a direction obtained by rotating the direction of the top of the screen by an angle between the direction of the ultrasound emitted by the ultrasound transmitter measured by the posture measurement unit and a vertical downward direction . An information processing system comprising: the information processing device according to claim 1 ; the ultrasonic transmitter; and the ultrasonic transducer. The apparatus further includes an imaging device for acquiring a contrast image of the biological tissue,
the information processing device is communicably connected to the photographing device,
The control unit causes the display unit to display the contrast image acquired by the imaging device together with the frame image .
6. The information processing system according to claim 5. An information processing method of an information processing device, which is communicatively connected to an ultrasonic transducer that receives a first ultrasonic wave emitted from an ultrasonic transmitter from a body surface toward inside the body and a second ultrasonic wave radially transmitted and a reflected wave of the second ultrasonic wave inside a living tissue, comprising:
A control unit of the information processing device
acquiring line data from the ultrasonic transducer, the line data being indicative of ultrasonic waves received by the ultrasonic transducer;
generating a frame image representing at least an inside of the biological tissue based on the intensity of the reflected wave represented by the line data;
displaying the generated frame image on a display unit;
When the first ultrasonic wave is indicated by the line data, displaying the frame image on a display unit together with an image indicating a direction in which the first ultrasonic wave is received by the ultrasonic transducer ;
Including,
The control unit is
determining a reference direction according to the attitude of the ultrasonic transmitter measured by an attitude measurement unit;
rotate the frame image and display it on the display unit so that a direction in which the ultrasonic transducer receives the first ultrasonic wave coincides with the reference direction;
Information processing methods. an ultrasonic transmitter that transmits a first ultrasonic wave from a body surface toward the inside of the body;
an ultrasonic transducer that receives a first ultrasonic wave transmitted by the ultrasonic transmission unit and radially transmits a second ultrasonic wave and receives a reflected wave of the second ultrasonic wave inside the biological tissue;
An information processing method for an information processing system including an information processing device communicably connected to the ultrasonic transducer,
A control unit of the information processing device
acquiring line data from the ultrasonic transducer, the line data being indicative of ultrasonic waves received by the ultrasonic transducer;
generating a frame image representing at least an inside of the biological tissue based on the intensity of the reflected wave represented by the line data;
displaying the generated frame image on a display unit;
When the first ultrasonic wave is indicated by the line data, displaying the frame image on a display unit together with an image indicating a direction in which the first ultrasonic wave is received by the ultrasonic transducer ;
Including,
The control unit is
determining a reference direction according to the attitude of the ultrasonic transmitter measured by an attitude measurement unit;
rotate the frame image and display it on the display unit so that a direction in which the ultrasonic transducer receives the first ultrasonic wave coincides with the reference direction;
Information processing methods. A computer program for a computer, the computer program being communicatively connected to an ultrasound transducer that receives, inside a biological tissue, a first ultrasound wave emitted by an ultrasound transmitter from a body surface toward inside the body, and a second ultrasound wave that is radially transmitted and a reflected wave of the second ultrasound wave, the computer program comprising:
acquiring line data from the ultrasonic transducer, the line data indicating the ultrasonic waves received by the ultrasonic transducer;
generating a frame image representing at least the inside of the biological tissue based on the intensity of the reflected wave represented by the line data;
A process of displaying the generated frame image on a display unit;
When the first ultrasonic wave is indicated by the line data, a process of displaying the frame image on a display unit together with an image indicating a direction in which the first ultrasonic wave is received by the ultrasonic transducer ;
causing the computer to execute a process including
determining a reference direction according to the attitude of the ultrasonic transmitter measured by an attitude measurement unit;
rotate the frame image and display it on the display unit so that a direction in which the ultrasonic transducer receives the first ultrasonic wave coincides with the reference direction;
Computer program.

Images