JP2023099261A - Medical information provision device, ultrasonic ct imaging device, and medical information provision system - Google Patents

Abstract

To reduce the possibility of false positive determination in breast cancer determination, and achieve improvement of examination accuracy and efficiency, and reduction of a burden on a subject in a breast cancer examination by mutually sharing information appropriately while executing imaging by a plurality of modalities and executing determination by a plurality of healthcare workers.SOLUTION: A medical information provision device includes: a display position calculation unit 51 for acquiring first image information on a subject imaged by a first medical image diagnostic device 1, acquiring second image information on the subject imaged by a second medical image diagnostic device 2 prior to the first medical image diagnostic device 1, and executing calculation so as to execute positioning processing between the first image and the second image by mutually sharing position information on the first image information and the second image information in the first image and the second image; and a display control unit 52 for causing the calculated position information on the second image to be reflected in the display of the first image and displayed.SELECTED DRAWING: Figure 5

Description

The embodiments of the present invention relate to a medical information providing apparatus, an ultrasonic CT imaging apparatus, and a medical information providing system.

For suspicion of breast cancer, for example, in examinations, examinations using a mammography apparatus, so-called mammography, are often performed. Then, based on the obtained X-ray image, medical professionals such as doctors and technologists interpret the image, and a report (hereinafter referred to as "interpretation report" as appropriate) stating findings (judgment) for suspected breast cancer. is created. A diagnostician makes a final diagnosis by referring to the interpretation report.

In this way, the presence or absence of breast cancer is determined based on two-dimensional images taken by mammography, but if the determination is made by only one medical worker, breast cancer (tumor) may be overlooked. Therefore, in order to prevent oversight, it is recommended that a plurality of doctors interpret and judge the same image at different timings. Such interpretation is referred to as multiple interpretation, but in addition to the case where there are multiple interpreting doctors who interpret the images as multiple doctors, the diagnostician makes the final decision based on the judgment made by one interpreting doctor. It also includes making a proper diagnosis.

On the other hand, when mammography is used, both mammary glands and tumors appear white, which makes it difficult to detect tumors. In particular, in the case of a breast with a high density of mammary glands, which is called a dense breast (dense breast), the area of the mammary gland that appears white is wide, so the sensitivity to tumors on the image decreases. Tumors can become more difficult to detect.

Therefore, in addition to examinations using mammography, determination using a different medical image diagnostic apparatus (hereinafter referred to as a "modality" as appropriate) is being recommended. Modalities used in this case include, for example, a handheld general-purpose ultrasonic diagnostic imaging apparatus.

A handheld general-purpose ultrasound imaging apparatus obtains an image by pressing a handheld ultrasound probe against the breast of a lying subject. Unlike mammography, the general-purpose ultrasonic diagnostic imaging apparatus does not need to crush the breast to be examined and does not use X-rays, so it is minimally invasive to the subject.

Further, in recent years, not only such handheld general-purpose ultrasonic diagnostic imaging apparatuses but also ultrasonic CT imaging apparatuses capable of obtaining ultrasonic tomographic images of the entire breast have come to be used. That is, in the above-described handheld general-purpose ultrasonic diagnostic imaging apparatus, the area to be imaged is strictly limited to the area including the area in contact with the ultrasonic probe and its vicinity. On the other hand, an ultrasonic CT imaging apparatus can acquire an entire breast image as a three-dimensional image.

However, when a plurality of different modalities are used for breast cancer examination, the acquired images, such as the dimension and the type of image such as whether or not it is a tomographic image, differ depending on each modality. Therefore, for example, when an interpreter compares images acquired by a plurality of modalities, he/she must understand the characteristics of each modality before interpreting the images.

Therefore, it is necessary to share information between different images. In addition, when images of the same subject are interpreted and judged by a plurality of radiologists, for example, if annotations are attached to the images, the information including the position information is shared among the plurality of images. Without it, an effective judgment cannot be made even if the image is interpreted.

Sharing of information among multiple modalities is disclosed in Patent Document 1 below. The patent document 1 mainly discloses information sharing between mammography that acquires three-dimensional images and images obtained by a handheld general-purpose ultrasonic diagnostic imaging apparatus using an ultrasonic probe.

JP 2015-164516

However, especially in the case of a hand-held type general-purpose ultrasonic diagnostic imaging apparatus, the area to be imaged is limited to the area including and near the area in contact with the ultrasonic probe as described above. On the other hand, in mammography, the entire breast area is imaged. Given this, in terms of the area in which the images are acquired, there is a Information will be shared.

Here, when the suspicion of breast cancer is determined based on the images taken by mammography, in general, about 5% of the total number of examinations requires further detailed examinations and biopsies. It is In addition, in view of the characteristics of mammography described above, the risk of oversight increases and sensitivity decreases according to the mammary gland density of the subject.

On the other hand, many of the findings detected by X-ray mammography, such as calcification, do not lead to lesions requiring treatment. Such findings can lead to false positives that result in negative results in detailed examinations, leading to a decrease in the specificity of screening as a whole.

The present invention reduces the possibility of false positive determination in breast cancer determination by appropriately sharing information with each other while performing imaging with a plurality of modalities and determination by a plurality of medical professionals. It is an object of the present invention to provide a medical information providing apparatus, an ultrasonic CT imaging apparatus, and a medical information providing system capable of improving examination accuracy and efficiency in medical examinations and reducing the burden on subjects.

A medical information providing apparatus according to an embodiment of the present invention acquires first image information relating to a breast of a subject photographed by a first medical image diagnostic apparatus, and prior to the first medical image diagnostic apparatus, Obtaining second image information about the breast of the subject photographed by a second medical image diagnostic apparatus, and obtaining position information and second image information of the first image information in the first image and the second image are shared with each other, the display position calculation unit performs calculations so that the alignment processing of the first image and the second image can be performed, and the position information of the second image calculated by the display position calculation unit is and a display control unit that reflects and displays the first image.

In addition, the display control unit in the medical information providing apparatus reflects the position information of the second image on the display of the first image based on the instruction of the radiogram interpreter who uses the medical information providing apparatus to make a judgment on the subject. to display.

Also, the subject is a breast, and the first medical image diagnostic apparatus for capturing the first image information acquired by the medical information providing apparatus is an ultrasonic CT imaging apparatus.

Furthermore, the second medical image diagnostic device for capturing the second image information acquired by the medical information providing device is either a mammography device, an ultrasonic CT imaging device, or an ultrasonic diagnostic imaging device. be.

The medical information providing apparatus is characterized in that the second image is a two-dimensional image, the first image is a three-dimensional image, and the characteristic region of the second image is displayed in the first image. .

Further, in the medical information providing apparatus, at least one of the first image and the second image further includes a schema indicating the part of the subject.

The display position calculation unit in the medical information providing apparatus is further characterized by displaying the rectal region position in the schema in the interpretation report.

Further, in the medical information providing apparatus, at least one of the first image and the second image further includes position information of a plurality of characteristic regions.

An ultrasonic CT imaging apparatus according to an embodiment of the present invention acquires image information related to an image of a subject, acquires position information related to a preceding image related to the subject that was previously captured, and obtains, in the image and the preceding image, By sharing the position information of the image information of the image and the position information of the image information of the preceding image with each other, the display position calculation unit performs calculation so that the image and the preceding image can be aligned, and the display position calculation unit a display control unit that reflects the calculated position information of the preceding image on the display of the image to display the image.

Further, the medical information providing system according to the embodiment of the present invention includes: a first medical image diagnostic apparatus for acquiring first image information about the breast of the subject; a second medical image diagnostic apparatus that acquires second image information relating to the breast of the patient; a providing device.

Also, the first medical image diagnostic apparatus that captures the first image information acquired by the medical information providing apparatus in the medical information providing system is an ultrasonic CT imaging apparatus.

Further, the medical information provided by the medical information providing apparatus to the radiogram interpreter in the medical information providing system is the first image information sharing the position information of the second image information.

With such a medical information providing apparatus, ultrasonic CT imaging apparatus, or medical information providing system according to the embodiment of the present invention, imaging with a plurality of modalities and determination by a plurality of medical professionals are performed, and mutual By appropriately sharing information, it is possible to reduce the possibility of false positive judgment in breast cancer diagnosis, improve the accuracy and efficiency of breast cancer screening, and reduce the burden on examinees. can.

1 is a configuration diagram showing the overall configuration of a medical information providing system according to an embodiment of the present invention; FIG. 1 is an external perspective view showing an ultrasonic CT imaging apparatus according to an embodiment of the present invention; FIG. 1 is a perspective view showing a structural overview of a measurement module in the configuration of an ultrasonic CT imaging apparatus according to an embodiment of the present invention; FIG. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram showing an excerpt from an examination of a subject using an ultrasonic CT imaging apparatus according to an embodiment of the present invention; 1 is a block diagram showing the internal configuration of a medical information providing apparatus according to an embodiment of the present invention; FIG. 4 is a flow chart showing the flow of medical information provision in the embodiment of the present invention. 4 is a flow chart showing the flow of medical information provision in the embodiment of the present invention. FIG. 4 is an explanatory diagram showing an example of display on the display section when medical information is provided in the embodiment of the present invention; (a) and (b) are explanatory diagrams showing an example of display on the display unit when medical information is provided in the embodiment of the present invention. FIG. 4 is an explanatory diagram showing an example of display on the display section when medical information is provided in the embodiment of the present invention; FIG. 4 is an explanatory diagram showing an example of display on the display section when medical information is provided in the embodiment of the present invention; FIG. 4 is an explanatory diagram showing an example of display on the display section when medical information is provided in the embodiment of the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Configuration of medical information provision system]
FIG. 1 is a configuration diagram showing the overall configuration of a medical information providing system S according to an embodiment of the present invention. The medical information providing system S is configured by connecting each device constituting the medical information providing system S to the information communication network N. FIG. In the medical information providing system S shown in FIG. 1, a first medical image diagnostic apparatus 1, a second medical image diagnostic apparatus 2, a medical image storage system 3, and a medical information providing apparatus 4 are connected to an information communication network N. It is

The medical information provision system S is used by medical professionals to determine the presence or absence of breast cancer, the need for reexamination, etc., based on medical images of subjects obtained using modalities in breast cancer screening. be. Moreover, as described above, a plurality of modalities are connected to the medical information providing system S. image is used for the final determination.

However, as for the order of imaging for the subject, the second medical image diagnostic apparatus 2 precedes, and then the imaging using the first medical image diagnostic apparatus 1 is performed. Further, when the medical staff makes a final judgment based on the first image, the judgment based on the second image (previous image) captured by the second medical image diagnostic apparatus 2 has already been made. It is assumed that

In addition, the medical information providing system S connected to the information communication network N in FIG. 1 describes only the devices necessary for the medical staff to determine the presence or absence of breast cancer in the subject by performing image interpretation. are doing.

Therefore, devices connected to the information communication network N are not limited to these devices, and may be arbitrarily connectable to, for example, each system, device, or other modality built in a medical institution. . Also, the method of connection may be wired or wireless.

[Configuration of First Medical Image Diagnostic Apparatus]
The first medical image diagnostic apparatus 1 corresponds to an ultrasonic CT imaging apparatus in the embodiment of the present invention. An ultrasonic CT imaging apparatus, which is the first medical image diagnostic apparatus 1, scans the entire breast, which is a subject to be inspected, using, for example, ultrasonic transducers arranged in a ring, and performs ultrasonic CT imaging. A device for acquiring an image.

FIG. 2 is an external perspective view showing an ultrasonic CT imaging apparatus 1, which is the first medical image diagnostic apparatus 1 according to the embodiment of the present invention. In the following description, the ultrasonic CT imaging apparatus will also be denoted by reference numeral 1 as appropriate.

The ultrasonic CT imaging apparatus 1 includes a bed 11 and a measurement module 12 (not shown in FIG. 2) arranged inside the bed 11 . The bed 11 is a table on which a subject to be examined is placed. In the examination, the subject lies face down on the upper surface 111 of the bed 11 indicated by the XY plane so as to extend in the Y direction, which is the longitudinal direction of the bed 11, from head to toe.

Although not shown in particular, the bed 11 shown in FIG. 2 may be provided with an elevating function in consideration of convenience when the subject gets on and off the bed 11 . Moreover, the bed 11 may be inclined from the horizontal plane, and the upper surface 111 thereof may not be flat.

The bed 11 is provided with an opening 112 . The opening 112 is approximately the chest area in the Y direction when the subject is placed on the bed 11, and is formed to have a size such that the breast, which is the subject, can be inserted. Note that the position, size, and shape of the opening 112 are examples. Also, two openings 112 may be provided at positions respectively corresponding to the left and right breasts.

Inside the bed 11 and directly below the opening 112, a measurement module 12 for measuring the breast is arranged. FIG. 3 is a perspective view showing a structural outline of the measurement module 12 in the configuration of the ultrasonic CT imaging apparatus 1 according to the embodiment of the present invention.

Measurement module 12 includes bath 121 and ring array 122 . The bath 121 is a container into which a medium such as degassed water having good ultrasonic wave propagation properties is injected and held during measurement. Therefore, in the ultrasonic CT imaging apparatus 1 according to the embodiment of the present invention, when the subject lies face down on the bed 11, the bathtub 121 accommodates at least a part of one breast, which is the subject. be done.

Note that the measurement module 12 is provided with a water supply/drainage mechanism such as a valve (not shown) or a pipe connected to the tank, and water is supplied to/drained from the bathtub 121 by this mechanism.

The measurement module 12 may also include a position-specific sensor (not shown). In the ultrasonic CT imaging apparatus 1 according to the embodiment of the present invention, after the breast, which is the subject, is inserted into the bathtub 121, imaging is completed by automatic scanning of the measuring device. Therefore, the entire examination area of the subject needs to be arranged in the measurement module 12 in order to image the entire mammary gland region of the subject.

Regarding this point, as a method of specifying the initial position of imaging, there is a method of specifying the position based on information obtained from the biological surface tissue of the subject, such as the papilla, by a camera, laser scanner, or the like. . In addition, by arranging an ultrasonic sensor or the like for acquiring biological tissue information in the Z-axis direction of the subject on the bottom surface of the measurement module 12, information based on the biological surface tissue of the subject's breast, such as the pectoralis major muscle and ribs, can be obtained. It is also possible to acquire the position information of external tissues outside the breast. The position information includes coordinate information indicating the position and orientation of the slice described in the DICOM header.

With such a position measurement sensor, by acquiring information based on the biological surface tissue of the breast, which is the subject, such as the pectoralis major muscle and ribs, and information on external tissue outside the breast, the relative position can be calculated and used by other It can also be used for alignment with modalities.

The ring array 122 is a measuring device for measuring biological information of a subject. The ring array 122 includes a plurality of measurement ultrasonic elements arranged in a plane (in the XY plane in the example of the ring array 122 shown in FIG. 3) so as to surround at least part of the subject. The ring array 122 is a group of ultrasonic elements that radiate ultrasonic waves into the bathtub 121 and receive scattered ultrasonic waves.

The ring array 122 converts electrical signals into ultrasonic waves, generates ultrasonic beams while adjusting phases and the like from a plurality of transducers, and emits the ultrasonic beams toward the breast, which is the subject. Then, it receives a plurality of scattered ultrasound waves from the breast, converts each of the scattered ultrasound waves into an electric signal, and outputs the electric signal. By performing these processes, cross-sectional information of the tissue inside the breast on the arrangement plane (XY plane) of the ultrasonic element group is obtained. This is called slice data.

Here, the scattered ultrasonic waves include at least one of reflected waves and transmitted waves. That is, the ring array 122 may be of a type that receives reflected waves, a type that receives transmitted waves, or a type that receives reflected waves and transmitted waves.

The ring array 122 is fixed inside the bathtub 121 by a support member 123 so as to be movable in a predetermined direction. Further, the support member 123 has a moving mechanism 124 . In the example of the measurement module 12 shown in FIG. 3, the moving mechanism 124 represents a sliding mechanism for moving the ring array 122 in the depth direction (Z direction).

Then, by performing ultrasonic measurement while moving the ring array 122 using the moving mechanism 124 and repeatedly acquiring information on different positions of the imaging cross sections, information on the entire tissue inside the breast, which is the subject, can be obtained. obtain. Here, the scanning interval is, for example, 1 mm. Therefore, when the depth of the measurement range is 20 mm, a total of 21 tomographic images are obtained. The direction of scanning may be from top to bottom in the Z direction or from bottom to top, but is preferably in one direction.

Also, the scanning direction need not be perpendicular to the imaging plane. For example, the movement mechanism 124 may be configured so as to perform a combination of rotational movement and parallel movement, such as spiral movement. In other words, it is sufficient that ultrasonic waves can be emitted so as to cover the inspection target range of the breast that is the subject suspended in the bathtub 121 .

Therefore, if the emitted beam can be controlled, the "movement" of the ring array 122 may be the movement of only the measurement position by controlling the emission direction of the ultrasonic beam. does not necessarily involve physical movement of

The shape of the ring array 122 may be circular, elliptical, or polygonal. Further, the ring array 122 may be an integral body, or may be a unit composed of a single element or a plurality of elements arranged in a predetermined shape such as a ring.

Furthermore, the planar shape of the ring array 122 may be semicircular or C-shaped instead of annular. Alternatively, it may be a linear array in which elements are arranged in a straight line. If the array structure is not arranged all around the subject, slice data may be obtained by applying ultrasonic waves from all directions, for example, by horizontally rotating the measuring device 12 . Furthermore, by moving this in the direction perpendicular to the plane (Z direction), volume data can be obtained.

Further, although the ring array 122 is used as the measuring device 12 in the embodiment of the present invention, the measuring device 12 only needs to be capable of capturing an ultrasonic three-dimensional image of the entire breast. Therefore, for example, it may be a measuring device capable of picking up a three-dimensional image by electronic scanning by arranging ultrasonic transducers around the object in a hemispherical shape.

FIG. 4 is an explanatory diagram showing an excerpt from an examination of a subject using the ultrasonic CT imaging apparatus 1 according to the embodiment of the present invention. As shown in FIG. 4 , the examination is performed in a prone position so that at least a part of one breast of the subject to be examined is inserted into the opening 112 .

That is, the ultrasound CT imaging apparatus 1 scans the ring array 122 in the Z direction while the breast, which is the object of measurement of the subject, is suspended in the bathtub 121 to collect data of the entire breast to be examined. .

In the case of the ultrasonic CT imaging apparatus 1 according to the embodiment of the present invention, measurement is performed for each breast on one side. Therefore, as shown in FIG. 2, when only one opening 112 is provided and it is necessary to measure both breasts, after completing the measurement of one breast, the subject should correct his/her posture and open the other breast. The breast of the woman is inserted into the bath tub 121 .

The main structure related to the examination of the ultrasonic CT imaging apparatus 1, which is the first medical image diagnostic apparatus, is as described above. A processing unit, an image processing unit, a drive control unit, and a control unit that controls each unit may be provided.

Under the control of the control unit, the transmission unit generates a drive signal for generating ultrasonic waves in the ring array 122, that is, a drive pulse that is an electric pulse signal to be applied to each piezoelectric transducer, and transmits the drive pulse to the ring array. 122. The transmission section includes, for example, a reference pulse generation section, a delay control section, a driving pulse generation section, etc., which are not shown, and each section performs the functions described above.

Also, the receiver receives reflected signals or transmitted signals received by the ring array 122 . Then, it performs phasing addition on the received signal, and outputs the signal obtained by the phasing addition to the signal processing unit.

The transmission/reception control unit determines a transmission aperture that contributes to transmission and a plurality of reception apertures that contribute to reception from among the plurality of ultrasonic elements forming the ring array 122 . After the ultrasonic waves transmitted from one transmission aperture are received by multiple reception apertures, the reception is repeated while shifting the position of the transmission apertures, thereby irradiating the subject with ultrasonic waves from all directions and forming a ring array. Slice data of the cross section on the 122 arranged planes is acquired.

The signal processing section generates various data using the received signal from the ring array 122 supplied from the receiving section, and performs processing for outputting the data to the image processing section and the control section. The signal processing section has, for example, a B mode processing section (or Bc mode processing section), a Doppler mode processing section, a color Doppler mode processing section, etc., which are not shown.

The B-mode processor visualizes the amplitude information of the received signal and generates data based on the B-mode signal. The Doppler mode processing unit extracts a Doppler shift frequency component from the received signal, and further performs FFT (Fast Fourier Transform) processing and the like to generate Doppler signal data of blood flow information. The color Doppler mode processor visualizes blood flow information based on the received signal and generates data based on the color Doppler mode signal.

The image processor generates a two-dimensional or three-dimensional ultrasound image of the scan area based on the data supplied from the signal processor.

For example, the image processing unit generates a two-dimensional slice image of the scan section from the supplied slice data. Then, three-dimensional ultrasonic image data is generated from the generated slice data by MPR processing (multiplanar reconstruction method) or volume rendering processing. The image processing unit then outputs the generated two-dimensional or three-dimensional ultrasound image to the display. Ultrasonic images include, for example, B-mode images, Doppler-mode images, color Doppler-mode images, and M-mode images.

The drive control section controls driving of the measurement module 12 . That is, the ring array 122 is slid by moving the moving mechanism 124 in the depth direction (Z direction). Also, the control unit comprehensively controls each unit of the ultrasonic CT imaging apparatus 1 . For example, an ultrasonic image generated in the image processing circuit is displayed on the display.

Returning to FIG. 1, the second medical image diagnostic apparatus 2 is used for breast cancer screening, for example, prior to photographing the subject's breast using the first medical image diagnostic apparatus 1. It is a medical image diagnostic apparatus used when imaging the breast as a result.

Before the subject undergoes examination using the first medical image diagnostic apparatus 1, the subject undergoes examination using the second medical image diagnostic apparatus 2. FIG. In other words, the second image is acquired at a different timing than the first image, and is often interpreted by a different medical professional than the first image. Further, when the medical staff has once completed the interpretation of the second image, an interpretation report is created based on the results of the interpretation, and the second image and the Linked and saved.

Here, the second medical image diagnostic apparatus 2 according to the embodiment of the present invention is either a mammography apparatus (mammography), an ultrasonic CT imaging apparatus, or a general-purpose ultrasonic diagnostic imaging apparatus.

The ultrasonic CT imaging apparatus 1 may be the same as the ultrasonic CT imaging apparatus 1 used as the first medical image diagnostic apparatus 1, or may be another ultrasonic CT imaging apparatus. For example, in regular breast cancer checkups, postoperative follow-up examinations, and the like, previously captured images of the same subject can be used.

Ultrasound imaging has no risk of radiation exposure associated with X-ray imaging, so there is no limit to the frequency of examinations. In addition, since an image can be captured without deforming the subject in a drooping state, the reproducibility of the imaged subject is high. Therefore, an examination using the ultrasonic CT imaging apparatus 1 as the second medical image diagnostic apparatus 2 is effective for observing temporal changes in the subject.

Therefore, according to the ultrasonic CT imaging apparatus 1 described in the embodiment of the present invention, specific parts of images acquired by the ultrasonic CT imaging apparatus 1 in the past, and findings and features detected during interpretation of these images can be aligned with the first image (inspection image) to suggest temporal changes and differences, thereby improving diagnostic efficiency. In addition, unlike general-purpose ultrasonic diagnostic equipment, image data of the entire subject can be acquired and stored, so diagnosis including the spread of new lesions can be facilitated.

Here, the general-purpose ultrasonic diagnostic imaging apparatus refers to a handheld ultrasonic imaging apparatus other than the ultrasonic CT imaging apparatus 1 described above, which is equipped with an ultrasonic probe that is used in contact with the breast, which is the subject, during examination. Refers to diagnostic equipment.

As an embodiment of the present invention, a case where a breast X-ray image diagnostic apparatus (mammography) is used as the second medical image diagnostic apparatus 2 will be described below as an example.

A medical image storage system 3 stores a first image or a second image of a subject's breast imaged by the first medical image diagnostic apparatus 1 or the second medical image diagnostic apparatus 2 . In addition, medical staff may interpret and judge these images, and the findings and interpretation reports summarizing the results may be stored together.

The medical image storage system 3 includes, for example, a storage medium and a storage device such as a hard disk drive (HDD) and a server. Alternatively, a medical image management system (PACS: Picture Archiving Communication System), which is a management system built in a medical institution, can be cited.

A hospital information management system (HIS) or a radiology information system (RIS) may also be used as the medical image storage system 3 .

Here, the first image information and the second image information acquired by the first medical image diagnostic apparatus 1 or the second medical image diagnostic apparatus 2 are stored in the medical image storage system 3 as described above. Saved. However, the first image information and the second image information are not stored in the medical image storage system 3, and are stored in the respective first medical image diagnostic apparatus 1 or the second medical image diagnosis acquired as appropriate. It may be stored inside the device 2 .

[Internal Configuration of Medical Information Providing Apparatus]
The medical information providing apparatus 4 is a first medical image diagnostic apparatus, which is an ultrasonic CT imaging apparatus 1. Based on the first image of the breast, which is the subject, which is captured and acquired, a medical professional interprets and judges the image. A device used to perform Then, the medical information providing apparatus 4 is used to create an interpretation report, which is the determination result. This interpretation report is used to provide necessary medical information when a doctor finally makes a diagnosis.

Therefore, as the medical information providing apparatus 4, for example, a PC (personal computer), a work station, or the like is used, and it may be a general-purpose machine or a special-purpose machine. In addition, it may be a stationary type or a portable type such as a smartphone, a tablet, or a PDA (Personal Digital Assistant). be.

FIG. 5 is a block diagram showing the internal configuration of the medical information providing apparatus 4 according to the embodiment of the invention. Note that FIG. 5 shows only the configuration necessary for medical staff to interpret and determine the presence or absence of breast cancer in a subject using the first image or the second image. However, the configuration of the medical information providing apparatus 4 is not limited to the depicted configuration, and may have other configurations.

The medical information providing apparatus 4 has a configuration in which a CPU (Central Processing Unit) 41, a ROM (Read Only Memory) 42, a RAM (Random Access Memory) 43, and an input/output interface 44 are connected via a bus B. there is Further, the input/output interface 44 is connected with an input section 45 , a display section 46 , a communication control section 47 , a storage section 48 and a control section 49 .

The CPU 41 reads and executes a boot program for activating the medical information providing apparatus 4 from the ROM 42 based on an input signal from the input unit 45 and reads various operating systems stored in the storage unit 48 .

Further, the CPU 41 reads programs and data stored in the RAM 43, the storage unit 48, etc., and loads them into the RAM 43. Based on the command of the program read out from the RAM 43, for example, the second image information is transferred to the first image information. It is a processing device that realizes a series of processes such as calculation and processing of data necessary to reflect image information.

The input unit 45 is configured by an input device such as a touch panel for inputting various operations by the medical staff using the medical information providing apparatus 4 . An input signal is created based on the operation of the medical staff via the input unit 45 and transmitted to the CPU 41 via the bus B. FIG.

The input unit 45 may be, for example, a GUI, a button, a keyboard, a trackball, a touch panel displayed on the display unit 46 described below, or a touch panel using a display separate from the display unit 46. device can be used.

In addition, devices provided in the medical information providing apparatus 4, such as a microphone and a camera, or various devices connectable to the medical information providing apparatus 4 can also be cited as a type of the input unit 45.

The display unit 46 displays, for example, an ultrasonic image generated by the image processing unit of the ultrasonic CT imaging apparatus 1 or an operation screen (for example, a GUI (Graphical User Interface) for receiving various instructions from the user). , etc. are displayed according to the control of the control unit 49 . That is, the display unit 46 receives an output signal from the CPU 41 via the bus B, and displays, for example, the first image, the second image, an interpretation report, or the processing result of the CPU 41 .

Specifically, the display unit 46 is, for example, a monitor, and as its type, it is possible to use a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

In the embodiment of the present invention, as shown in FIG. 5, the input unit 45 and the display unit 46 are described as one component of the medical information providing apparatus 4, but the configuration is not limited to this. do not have. Further, as the display unit 46, for example, one monitor may be used, or a plurality of monitors may be used to appropriately display necessary information.

The communication control unit 47 is means such as a LAN card and a modem, and is a means for connecting the medical information providing apparatus 4 to an information communication network N such as the Internet or LAN. Data transmitted/received to/from the information communication network N via the communication control unit 47 is transmitted/received to/from the CPU 41 via the input/output interface 44 and the bus B as an input signal or an output signal.

The standard for information and medical images exchanged with other devices via the communication control unit 47 and the information communication network N may be any standard such as DICOM (Digital Imaging and Communication in Medicine).

Based on an instruction from the display position calculation unit 51, which will be described later, the communication control unit 47 obtains a first image of the breast, which is the subject, captured by the ultrasonic CT imaging apparatus 1, and a mammogram obtained before the first image. Call image information such as a second image such as . The interpretation report for the second image can also be called up if there are still diagnostic results for the second image.

The interpretation report includes patient's basic information and findings information. Finding information includes, for example, finding information on lesions (positions and types of suspected lesions, etc.), mammary gland density determination results, annotations and regions of interest (ROI: Region Of Interest) that serve as the basis for findings in diagnostic images (hereinafter referred to as "annotation etc.) is included.

In addition, the learning result of CAD (Computer-Aided Detection) used in interpretation of the second image can be included. These annotations and the like may be called up together with the image data of the second image. Hereinafter, images and their annotations are collectively referred to as image information.

The storage unit 48 is composed of a semiconductor or a magnetic disk, and stores programs and data to be executed by the CPU 41, for example, for reflecting the position information in the second image information to the first image information. .

The control unit 49 comprehensively controls each unit of the medical information providing apparatus 4 . For example, the control unit 49 acquires the first image information to be interpreted or the like from the medical image storage system 3 or the ultrasonic CT imaging apparatus 1 based on instructions from the medical staff via the input unit 45. , is displayed on the display unit 46 .

The display position calculation unit 51 acquires first image information related to the breast, which is the subject, imaged by the first medical image diagnostic apparatus 1, and performs second image information prior to the first medical image diagnostic apparatus 1. Second image information about the breast of the same subject photographed by the medical image diagnostic apparatus 2 is acquired, and calculation is performed so that the position information of the second image can be shared in the first image.

As a result of aligning the second image and the first image, a radiogram interpreter of the first image captured by the ultrasonic CT imaging apparatus 1, when performing radiogram interpretation based on the first image, for example, the first The second image and its findings can be reflected on the image.

As a method of reflection, the image of the relevant portion of the second image, the annotation of the relevant portion, etc. are superimposed on the first image. In addition, the range of a site that is likely to be overlooked in the second image (for example, a high-density mammary gland region in which mammography has low sensitivity) may be detected in the first image and the corresponding area may be suggested.

In this manner, in ultrasonic diagnosis using the ultrasonic CT imaging apparatus 1, the sensitivity of ultrasonic diagnosis is low, but the sensitivity of X-ray diagnosis is high. 2 can be obtained from mammography. In addition, in the high-density mammary gland region where the sensitivity of X-ray diagnosis decreases, ultrasonic diagnosis that can exert its power is performed. As a result, ultrasonic diagnosis can be performed in more detail. It is possible to improve diagnostic accuracy.

In addition, in the ultrasonic CT imaging apparatus 1 according to the embodiment of the present invention, when interpreting the first image, second image information such as a part of the previously captured second image and annotations added during interpretation is obtained. can also be reflected on the corresponding position of the first image, which contributes to the improvement of diagnostic efficiency and diagnostic accuracy of the radiologist.

A method of aligning the first image and the second image, such as reflecting the annotation of the second image on the first image, will be described below. The mammography used as the second medical image diagnostic apparatus 2 in the embodiment of the present invention compresses the breast, which is the subject, and generates a two-dimensional second image composed of XY axes.

On the other hand, the first image captured by the ultrasonic CT imaging apparatus 1, which is the basis for judgment and diagnosis by medical professionals, is three-dimensional information obtained by rendering two-dimensional slice information. Therefore, it is necessary to reflect the annotations and the like added to the two-dimensional image to the three-dimensional image.

Therefore, the display position calculation unit 51 grasps the position information about the second image including the annotation attached to the second image from the second image information, and calculates the positions of the first image and the second image. Align. Further, the display position calculation unit 51 performs display on the first image based on the position information so that the first image displayed on the display unit 46 can appropriately reflect the annotations and the like. Calculate the position.

For example, the second image information obtained by mammography includes positional information of the imaged breast or positional information such as annotations. As for the position information of the second image information, for example, the position information is stored in the header.

The display position calculation unit 51 grasps the depth from the papilla specified from the slice position as the Z-axis, for example, with the position of the papilla, which is a landmark, as a reference. Also, the XY plane is grasped by the concentric distance from the nipple.

Then, the display position calculation unit 51 converts the points in the second image represented two-dimensionally into lines on the first image. Especially in the case of mammography, since the breast to be imaged is deformed, the distortion is corrected based on multiple landmarks such as the nipple, mammary gland, base of the pectoralis major muscle, or the position of the ribs. Transformation processing of coordinates is performed on the basis of these human tissues.

An ultrasonic CT image is an image in which buoyancy is received at the time of imaging, while a mammography image is captured under pressure and stretched. Therefore, it is more preferable to deform the contours to correspond to each other. Certainly, even if the contour is corrected, the internal structure does not match up to the pixel unit, but if the dispersion or deviation of the local hardness is not large, it is possible to correct the overall shape and match it. , the internal structure can be easily compared between the ultrasound CT image and the mammography image. This is more effective when there are multiple lesions detected by each modality or when different imaging findings are presented, compared to when one lesion is detected by each modality.

Here, an example of a position calculation method for a more specific distortion correction process and alignment process technique by the display position calculation unit 51 will be described. The display position calculator 51 first calls the first image and the second image from the medical image storage system 3 through the communication controller 47 .

Next, the display position calculator 51 performs segmentation on the first image and the second image. For example, the papillary region, the tumor region, the epidermal region, the ribs, and the pectoralis major muscle are detected from the first image acquired by the ultrasonic CT imaging apparatus 1, that is, the ultrasonic CT image. On the other hand, from the second image, that is, the mammography image, the nipple area, the mammary gland area, the outline of the breast, and the mass are detected.

Next, the imaging directions of the ultrasound CT image and the mammography image are matched. Mammography images are usually captured from multiple imaging directions. Therefore, the imaging direction is not the same for each imaging. On the other hand, for ultrasonic CT images as well, the axis of imaging differs in the rotation direction around the nipple for each imaging. Therefore, the central axis of the ultrasonic CT image and the imaging direction of the mammography image (for example, the MLO direction (Medio-Lateral Oblique: medio-lateral oblique direction) and CC direction (Cranio-Caudal: cranio-caudal direction)). Based on the range information obtained, it is necessary to match the imaging directions of both images.

As a matching method, first, in an ultrasonic CT image, a projection image is generated with the direction from the papilla toward the back or the sagittal direction as an axis. At this time, a projected image is created ignoring the epidermis. Next, non-rigid deformation and rigid deformation techniques are used to deform the ultrasonic CT image so that it approximates the mammography image to be compared. Furthermore, while rotating the deformed ultrasound CT image and mammography image, the degree of similarity at multiple angles is evaluated. Then, the direction with the highest degree of similarity is determined as the shooting direction.

Next, the ultrasound CT image and the mammography image are also aligned in the depth direction. An ultrasound CT image is a circular cross-sectional slice from the chest wall toward the papilla. Therefore, for example, the depth of a slice in which a region of interest such as a tumor region exists is specified from a mammography image.

Therefore, first, MPR slice images including a tumor are generated at multiple depths in ultrasonic CT images. Next, the degree of similarity in shape between the tumor region of the ultrasound CT image and the tumor region of the mammography image at each depth is evaluated. Then, the slice with the highest degree of similarity is identified as the depth-matched slice.

The MPR slice matched in the previous step becomes the position and orientation of the mammography image. Further, by aligning the positions of each tumor region in the ultrasound CT image and the mammography image, fine adjustment of translation within the MPR slice is performed. In addition to tumor masses, linear structures such as Cooper's ligaments and mammary glands can also be used as landmarks.

Here, the method of aligning the position of the ultrasound CT image, which is the first image, with the position of the mammography image, which is the second image, has been described. good.

The display control unit 52 displays the second image that has been aligned with the first image based on the position information calculated by the display position calculation unit 51 . At this time, the display control unit 52 can arbitrarily set how the display unit 46 displays the first image and the second image.

In addition, the display control unit 52 displays the annotations and the like displayed in the second image on the first image based on the calculated position information. As a result, the content of the interpretation report based on the second image captured by the second medical image diagnostic apparatus 2 that has already been created even in the first image that is interpreted by the medical staff and used as a basis for judgment can be reflected to perform interpretation and judgment. Further, in creating an interpretation report for the first image, the position information of the lesion information on the first image calculated by the position calculation unit is automatically reflected in the interpretation report.

The positional information of the lesion information detected in the first image is, for example, the three-dimensional positional information of the first image, the positional information on the YX plane in the annular plane of the subject, and the depth from the papilla to the chest wall. The position of the Z plane in the longitudinal direction can be indicated by a slice number or a distance from the papilla.

It is also possible to calculate the positions corresponding to the CC plane and MLO plane of mammography, reflect them in the schema of the ultrasound CT image (a schematic diagram of the subject showing finding information), and display them.

Since the MLO plane of mammography is not parallel to the trunk but tilted, it has a tilt from the Y-axis of the annular plane of ultrasonic CT. Therefore, an ultrasonic CT image may display auxiliary lines indicating the CC plane and the MLO plane. Furthermore, when the slice data position is changed or the body axis is displayed with an inclination, dynamic information may be displayed in which auxiliary lines and contour lines move together with the ultrasonic CT image.

In this way, by displaying the tomographic plane, it becomes possible for a radiologist accustomed to the position display of conventional mammography to recognize the positional information of the lesion without feeling discomfort even in the ultrasonic CT image. It is possible to reduce the burden of image interpretation and report creation on the person.

In addition, it is also possible to display an input section for error adjustment in consideration of the error on the screen, and it is also effective to display the auxiliary line with a width in consideration of the estimated error. As described above, such display is also more difficult when there are multiple or different imaging findings, and when grasping the positional relationship between mammary glands and lesions, compared to when each modality has a single and identical imaging finding. It is valid.

The information communication network N connects the first medical image diagnostic apparatus 1, the second medical image diagnostic apparatus 2, the medical image storage system 3, and the medical information providing apparatus 4, respectively. By being connected to the information communication network N, these devices are enabled to exchange information about images acquired by the first medical image diagnostic apparatus 1 or the like. Examples of the information communication network N include a network such as the Internet and a LAN (Local Area Network).

[motion]
Next, based on the second image information acquired by the second medical image diagnostic apparatus 2, the medical information providing apparatus 4 transmits the position information to the first image information acquired by the first medical image diagnostic apparatus 1. The flow of reflecting on the image will be described with reference to FIGS. 6 and 7. FIG. 6 and 7 are flow charts showing the flow of providing medical information according to the embodiment of the present invention.

In addition, while explaining the above flow, the contents displayed on the display unit 46 will also be explained with reference to FIGS. 8 to 10. FIG. 8 to 10 are explanatory diagrams showing examples of display on the display unit 46 when medical information is provided in the embodiment of the present invention.

Note that, as described above, a medical professional who is a radiogram interpreter who determines the presence or absence of breast cancer in the breast, which is the subject, makes a determination based on the first image captured by the ultrasonic CT imaging apparatus 1 . On the other hand, whether or not the second image acquired by mammography (the second medical image diagnostic apparatus 2) or the second image information such as the annotation is referred to or not is determined by the medical staff who makes the determination, as will be described later. preferably entrusted to

That is, when performing a combined diagnosis with different modalities, it is possible to improve sensitivity while reducing false positives or false negatives caused by modalities by performing diagnosis without referring to the results of other modalities. .

On the other hand, in order to improve the specificity of previously diagnosed mammography diagnostic results, there is a demand for diagnosis focused on ultrasound CT images based on previous diagnostic results. In this case, it is more efficient to display the finding information of the mammography image in advance on the display unit 46 on which the ultrasonic CT image is displayed.

First, the medical staff selects a subject to be judged, and inputs, for example, a patient ID or the like for identifying the subject to the medical information providing apparatus 4 (ST1). Although not shown in FIG. 6, since the medical information is touched, the ID of the medical worker is entered in advance, and the access authority is confirmed.

The medical information providing apparatus 4 detects the acquisition instruction of the first image information captured by the ultrasonic CT imaging apparatus 1 input from the input unit 45 by the medical staff (ST2), and transmits the medical information via the communication control unit 47. Access the image storage system 3 . Then, the first image information about the subject to be judged and diagnosed is acquired from the medical image storage system 3 (ST3).

As described above, the medical staff makes a determination based on the first image captured by the ultrasonic CT imaging apparatus 1 . Therefore, it can be considered that it is sufficient if only the first image is displayed on the display unit 46 .

However, referring to the second image information acquired in advance and the interpretation report created based on the second image is very important in trying to reduce false positives as much as possible regarding the presence or absence of breast cancer. Useful.

Therefore, whether or not to refer to the second image is ultimately left to the will of the medical staff, but the medical information providing apparatus 4 is requested by the medical staff to refer to the second image. In anticipation of this, it is desirable to acquire the second image information in advance.

Here, in the embodiment of the present invention, it is assumed that the mammography acquires the second image information in advance before the ultrasonic CT imaging apparatus 1 acquires the first image information. Therefore, the medical information providing apparatus 4 accesses the medical image storage system 3 based on, for example, the patient ID of the subject used when acquiring the first image. Then, the second image information acquired by mammography is searched in the medical image storage system 3 (ST4), and the corresponding second image information is acquired (ST5).

Note that the flow of acquiring the second image information after acquiring the first image information has been described so far. However, the acquisition order is not limited, and the first image information may be acquired after the second image information acquired first.

Further, here, acquisition of the second image information is automatically performed by the medical information providing apparatus 4 in response to an instruction to acquire the first image information from the medical staff. It is also possible to wait for an instruction to acquire the second image information from.

Further, here, an example has been described in which acquisition of the second image information is executed immediately after the acquisition of the first image information. However, with respect to the second image information obtaining process, for example, after the first image displayed on the display unit 46 is identified, a second image closer to the displayed first image is obtained, as will be described later. It is also possible to execute a process of extracting an image, performing position alignment, and displaying the image. In this case, as a method of obtaining and extracting the second image information, for example, executing matching processing with all the captured second images with respect to the specified first image, or Various methods can be employed, such as extraction taking into account the shooting direction.

The display position calculator 51 checks the position information included in the obtained second image information (ST6). Also, if location information such as annotations is included, the location information is also checked in the same way. As a result, if position information is included in the acquired second image information (YES in ST7), the display position calculator 51 identifies the position information (ST8).

Here, the ultrasonic CT imaging apparatus 1 can acquire a plurality of images in one scan. Therefore, when a medical professional determines the presence or absence of breast cancer, he or she compares the plurality of first images and selects an image suitable for the determination from among them.

Therefore, the display control unit 52 checks whether or not the slice data of the first image has been selected by the medical staff (ST9), and if the first image to be displayed is selected and specified, (YES in ST9), the selected first image is displayed on the display section 46 (ST10).

On the other hand, the medical information providing apparatus 4 is on standby until the medical staff selects the first image to be displayed (NO in ST9). Although not shown, after waiting, the medical staff performs an operation to switch slices, updates the slices one by one, and selects an image to be displayed. However, if the relevant first image does not exist, the medical staff can proceed to the next image by inputting that there is no relevant image, or perform the cancel operation to execute the position information specifying process again. can do.

FIG. 8 shows the display 46 with the first image selected by the medical personnel. Note that the display layout of each content including the first image shown in FIG. 8 is merely an example, and the layout, content, scale, and the like can be freely set.

A first image captured by the ultrasonic CT imaging apparatus 1 is displayed on the display unit 46 shown in FIG. In FIG. 8, the screen is largely divided into left and right, and the first image selected by the medical staff is displayed on the left side of the screen. In addition, the area surrounded by a square in the first image indicates an area such as ROI, which is of interest to the medical staff.

In the upper right part of the screen, nine slice images, which are ultrasonic CT images obtained by imaging the subject, are displayed here. Among the nine slice images, the upper left slice image is surrounded, indicating that this image is the image selected by the medical staff. The process of selecting one first image from a plurality of slice images is the process described in steps ST9 and ST10.

Furthermore, a body mark indicating the breast, which is the subject, is displayed in the lower right portion of the screen of the display unit 46 . Here, one body mark is shown for each of the left and right breasts, and the areas A to D are concentrically divided from the center toward the outside.

Note that, unlike the body mark used in general-purpose ultrasonic diagnostic imaging apparatuses, in the case of the ultrasonic CT imaging apparatus 1, a figure indicating the shoulder (armpit) is not displayed, and a complete circle is displayed. The rectangular area shown in the first image displayed on the left side of the screen is indicated by a round mark in the body mark on the right side.

By selecting and displaying the first image to be displayed on the display unit 46 in this manner, the first image to be interpreted that is of interest to the medical staff is specified. Therefore, the display position calculation unit 51 next calculates position information matching the selected first image based on the position information included in the second image information (ST11 in FIG. 7).

The processing in this display position calculation unit 51 is also processing automatically performed in the medical information providing apparatus 4 . During this time, the medical staff, for example, sees the first image selected by him/herself displayed on the display unit 46 and makes a determination.

While viewing the first image, the medical professional can at any time refer to a second image taken with a different modality of mammography on the same subject. There are two possible methods for referring to the second image.

The first is a case where the annotation or the like displayed on the second image is superimposed on the first image displayed on the display unit 46 for reference. As described above, when the first image is displayed on the display unit 46, the display position calculation unit 51 superimposes the second image information on the displayed first image. Secondly, the position information that can be reflected in the first image is calculated based on the position information in the second image.

Therefore, when there is an instruction from the medical staff to superimpose and display the second image information on the first image (YES in ST12), the display control unit 52 superimposes the second image on the first image. The displayed annotation or the like is superimposed on the first image and displayed (ST13).

FIGS. 9A and 9B show an example in which the annotations and the like displayed in the second image are superimposed on the first image displayed on the display unit 46 and displayed. In other words, the first image captured by the ultrasonic CT imaging apparatus 1 is superimposed with the annotations added by the medical staff to the second image captured by mammography.

More specifically, the display section 46 shown in FIG. 9 is divided into three areas. First, the first image selected and displayed by the medical staff is displayed on the left side of the screen. On the other hand, the right side of the screen is further divided into upper and lower parts. "MMG" is written on the upper side, and the image displayed here is the second image, that is, the image captured by the second medical image diagnostic apparatus 2, mammography.

In the display unit 46 shown in FIG. 9A, the image obtained by the MLO imaging is displayed on the left side, and the image obtained by the CC imaging is displayed on the right side. Annotations and the like are displayed in squares on images obtained by MLO imaging. That is, the annotations and the like added to the image obtained by mammography, which is the second image, are displayed in the ultrasound CT image, which is the first image, as a result of the registration of the first image and the second image. is also added to and displayed.

Also, two body marks are displayed below the second image, and similarly, the left side is the body mark for MLO imaging, and the right side is the body mark for CC imaging. In the body mark, the position of the annotation or the like displayed in the second image is indicated by a circle.

A slice image and a body mark captured by the ultrasonic CT imaging apparatus 1 are displayed in the lower right portion of the screen of the display unit 46 . The displayed contents here are the contents explained in FIG. Note that "RE" is displayed in the upper left part of this area, which is an abbreviation for "Ring Echo", that is, the ultrasonic CT imaging apparatus 1. FIG. c

By superimposing and displaying in this way, medical staff can refer to the finding information of images of the same subject taken with different modalities while using the first image as a basis. As a result, it is possible to make judgments after grasping all of a plurality of viewpoints that differ in terms of time and person. Therefore, it is possible to reduce false positives and improve the accuracy of determination in medical examinations.

It should be noted that here, the annotation added to the image obtained by mammography, which is the second image, is the result of alignment between the first image and the second image, and the ultrasound image, which is the first image. An example in which a CT image is also added and displayed has been described. However, on the contrary, the annotations and the like added to the ultrasonic CT image, which is the first image, are added to the image obtained by mammography, which is the second image, through the alignment process. It is also possible to display it.

Further, FIG. 9A shows a screen example in which a second image is also displayed on the display unit 46 in addition to the first image. However, instead of displaying both the first image and the second image in this way, it is also possible to display only the first image on which annotations and the like are displayed on the display unit 46 .

Further, FIG. 9(b) shows a plurality of imaging findings in the second image individually displayed in the first image. Mammography may provide clear imaging findings even when information around the mass is not clear on ultrasound CT images. By superimposing the imaging stations of other dalities on the ultrasound CT image in this way, the radiologist can more clearly determine the lesion.

On the other hand, if the medical staff does not want the position information such as the annotation shown in the second image to be superimposed and displayed on the first image (NO in ST12), the display control unit 52 Annotations and the like are not displayed on the first image displayed on the display unit 46 because no superimposition display instruction has been received from the medical staff.

Note that the description so far has been based on the premise that position information such as annotations is included in the second image information, but it is conceivable that the position information is not included in the first place. That is, this is the case where no annotation or the like is attached to the second image. In such a case (NO in ST7 of FIG. 6), the display position in the second image is not reflected in the first image. Therefore, the display position calculation unit 51 does not calculate the display position, and the process transitions between step ST13 and step ST14.

Further, the display control unit 52 confirms whether or not an instruction to display the second image has been received from the medical staff (ST14). Thus, a display method in which the display unit 46 displays the second image itself in addition to the first image is another method of referring to the second image. When the display control unit 52 receives the display instruction of the second image from the medical staff (YES in ST14), the display control unit 52 selects the second image corresponding to the first image, and displays the second image on the display unit 46. The second image is displayed simultaneously with the first image (ST15).

Here, to simultaneously display the first image and the second image means to display both images within the display area of the display unit 46 without overlapping. This point is different from the case where the annotation of the second image is superimposed on the first image and displayed.

FIG. 10 shows an example in which both the first image and the second image are displayed on the display section 46 . In the display unit 46 shown in FIG. 10, "RE", that is, the first image, which is the image of the ultrasonic CT imaging apparatus 1, is displayed on the left side of the screen. Lines are drawn in the vertical and horizontal directions of the screen as annotations and the like.

A mammography (MMG) image is displayed on the right side of the screen. The difference from the display example of FIG. 9 is that the display of the body mark and the display of the image related to the breast on one side are omitted here. Therefore, the second image can be displayed in a larger size. Also, annotations and the like are similarly drawn at positions corresponding to the positions of the annotations and the like displayed in the first image.

11 and 12 are display examples in which the positions corresponding to the CC plane and the MLO plane of mammography are calculated and displayed after being reflected in the schema of the ultrasound CT image (the schematic diagram of the subject showing the finding information). be. In this example, an error adjustment input section is displayed on the screen in consideration of an error occurring in the subject's posture and the angle of the imaging section when the first image and the second image are captured.

FIG. 11 shows the mammography image schema on the top right. Lesion locations are indicated by gray circles. The lower right diagram shows three-dimensional breast shape information, error input and display windows, and the left diagram shows an ultrasonic CT image. The lower right three-dimensional breast shape diagram shows a trihedral view, in which the lower left is a front view, the upper left is a schematic CC sectional view from above, and the lower right is a side view. On this three-dimensional breast contour diagram, the MLO section is indicated by a long dotted line, and the CC section is indicated by a short dotted line. Also, since each cross section has a thickness, the center line of the cross section is overlapped as a line indicating the central plane in the cross-sectional direction of the cross section having thickness. A band area perpendicular to each cross-sectional center line is the estimated lesion position.

If there is an error in the MLO angle of the mammography image, changing the position of the MLO will change the intersection of the two bands. Therefore, when the radiologist changes the tilt adjustment input of the MLO section of A and the error value of the display window, the estimated position in the ultrasonic CT image changes in an arc around the black circle. At this time, the estimated cross-sectional position does not change significantly.
On the other hand, FIG. 12 shows a display example in which the body axis of the ultrasonic CT image is tilted and there is a tilt error in the annular cross section. When the tilt adjustment input of the body axis of B and the display window error value are changed, the cross-sectional position of the ultrasonic CT image to be selectively displayed is changed. If the annular cross section is not recalculated, the estimated fault plane changes depending on the horizontal position. A new annular cross-sectional image obtained by slightly correcting the tilt of the body axis may be recalculated and displayed from a plurality of corresponding annular cross-sectional images.

By displaying the first image captured by the ultrasonic CT imaging apparatus 1 and the second image captured by mammography side by side in this way, it is possible to compare both more directly and overall. , making it easier to determine and diagnose based on the first image.

Of course, the instruction to display the second image on the display unit 46 may not arrive from the medical staff (NO in ST14), and the second image is not displayed on the display unit 46 in this case.

Next, the display control unit 52 confirms whether or not the first image displayed on the display unit 46 is to be selected as the image to be attached to the diagnosis report from the medical staff (ST16).

Here, the medical staff may wish to display an image different from the first image that has been displayed on the display unit 46 (NO in ST16). In this case, the medical staff selects another image acquired at the same timing as the first image currently displayed on the display unit 46, that is, taken in the same examination, as a new first image. (ST17).

Specifically, for example, the medical staff selects an image different from the image selected so far from the plurality of slice images shown on the right side of the screen in FIG. 1 image is selected, and the display control unit 52 receives an instruction to change the image (YES in ST17). In this case, the process returns to step ST9, and each process explained so far is repeated again.

On the other hand, if there is no instruction to select a new image as the new first image (NO in ST17), the first image that has been displayed on the display unit 46 so far will be used for the diagnosis report. .

The medical staff then prepares an interpretation report using the first image. In this case, the display control unit 52 transitions the screen and performs display for creating an interpretation report again (ST18).

As described above, in the medical information providing apparatus 4 of the present invention, position information is shared among a plurality of images captured in different formats, and position information in one image is reflected in the other image. Therefore, in determining breast cancer, imaging with multiple modalities, determination by multiple medical professionals, and appropriate mutual information sharing will reduce the possibility of false positive determinations and improve breast cancer screening. It is possible to improve inspection accuracy, increase efficiency, and reduce the burden on the subject.

Then, an interpretation report is created as a result of determination based on images captured by a plurality of modalities. Therefore, the diagnostician can finally diagnose the subject by referring to the extremely useful interpretation report.
<Other Embodiment 1>

An example of comparing the positions of the first image acquired by the ultrasonic CT imaging apparatus 1 and the second image, which is the image acquired by the mammography 2, has been described so far.

On the other hand, in clinical practice, medical professionals often use schemas to describe their findings in more detail. Here, the shuma is a schematic diagram in which the medical staff shows his/her own findings as described above. For example, in the screen example shown in FIG. The schematic diagram shown below can be mentioned.

When a schema is included as finding information included in the electronic medical record, the position and size of the lesion may be acquired more accurately as the finding information of the second image. Further, by creating a schema of the first image when interpreting the first image, interpretation report creation support can be added.

As an embodiment of the present invention, an example in which a schema is interposed in registration of the first image and the second image will be described. The display position calculation unit 51 acquires from the medical image storage system 3 a first image that is an ultrasound CT image as an object to be interpreted and a mammography image (second image) as a preceding image. Accompanying the acquisition of findings and interpretation report information. This interpretation report information includes schema images and lesion position information in terms of language. A mammography schema consists of a plurality of two-dimensional images including at least two types from MLO and CC sections.

In addition, the medical image storage system 3 also stores a template of a schema image in the ultrasonic CT image. The display position calculation unit 51 also acquires templates of these schema images. A schema image of an ultrasound CT image is represented by the depth direction from the papilla to the chest wall and the XY coordinates on the annular cross section. In addition, the obtained ultrasonic CT images and mammography images contain position information such as the number and number of slices, the angle at which mammography was taken, and the XY coordinates of the pixels of the reconstructed images.

Subsequently, the display position calculation unit 51 aligns the template of the schema image of the mammography and the schema image of the ultrasound CT image. The MLO axis of mammography has an inclination with respect to the trunk axis of the subject. On the other hand, in an ultrasonic CT image, the trunk is tilted with respect to the horizontal plane at the time of imaging, so the annular plane is not a completely horizontal cross section.

However, the CC section can be measured almost accurately in any image. Therefore, the CC section of mammography and the X-axis plane of the ultrasound CT image are aligned, the MLO of the captured image and the inclination of the annular plane are calculated, and the mammography schema is displayed on the schema image of the ultrasound CT image. Generate coordinates.

Subsequently, the display position calculation unit 51 acquires position information of mammography finding information (for example, image findings of a tumor) in the mammography schema image. Then, the position information on the mammography schema is reflected in the generated schema image of the ultrasound CT image to specify the position of the obtained mammography finding information. Note that the display position calculation unit 51 may generate a schema image from a mammography image instead of the schema information entered by a medical staff included in a mammography interpretation report.

Further, the corresponding slice data of the ultrasonic CT image is specified from the depth information of the schema image of the ultrasonic CT image. Also, the position of the mammography finding information on the ultrasound CT image is specified from the XY coordinates in the specified slice data, and displayed.

Note that if the interpretation report describes location information and finding features in language, this may be extracted by natural language processing technology and associated with the schema or the second image as an annotation. In these cases, the display position calculation unit 51 acquires the finding information in the schema image information or the language from the interpretation report saved in advance.
<Other Embodiment 2>

The discussion so far has not addressed the number or types of lesions detected. However, the number of lesions detected in one examination is not limited to one. Furthermore, there are multiple types of lesions that are detected. For example, cysts and mastitis, which are benign, and invasive carcinoma and lobular carcinoma, which are malignant. In diagnostic imaging, these are sometimes determined by imaging findings such as the shape of the margin and intensity of brightness. For example, the type of fiber line is detected in ultrasound CT images, but is difficult to identify in X-ray mammography. On the other hand, X-ray mammography has a high detection rate for microcalcifications. By comprehensively visualizing the imaging findings between these different modalities, false negatives and false positives in interpretation can be reduced.

An example of displaying imaging finding information of different modalities for each of a plurality of imaging findings will be described below. The image finding information is stored in the medical image information providing apparatus 2 as, for example, an image finding judgment table, and is acquired by the display position calculation unit 51 .

The image finding information includes the characteristics of each image of ultrasound CT images and mammography and their findings (marginal information, tumor shape, size, intensity of brightness, difficulty of evaluation, etc.). Major imaging findings include masses, calcifications, cooper's ligament spasms, cysts, and skin spasms. However, it is not limited to these, and may include, for example, a single tumor mass, spreading over a certain region such as microcalcification, or a tissue region such as a mammary gland region.

The display position calculation unit 51 individually detects the position information of the image finding from the mammography image that is the second image, assigns an identifier to the image finding, and stores it. When a plurality of imaging findings are detected, identification numbers are assigned, for example, in order from the one with the narrowest area. Image findings are also detected in the same manner in ultrasonic CT images.

Next, the display control unit 52 associates the features of the ultrasound CT images and the mammography images in the image finding determination table, the findings, and the image findings with identification numbers with each modality. The display control unit 52 displays the imaging findings according to the input instructions of the medical staff. The image findings are displayed by, for example, superimposing the ultrasound CT image findings on the ultrasound CT image and superimposing the mammography image findings on the mammography image. Alternatively, the image may be displayed superimposed on the schema of the ultrasound CT image or the schema of the mammography image.

Furthermore, one or more detected image findings may be simplified and displayed in each image or generated schema. As a display example of such imaging findings, an icon or color can be assigned to a specific imaging finding. In the following, for example, a case will be described in which an image finding is iconized and displayed for the schema of each image.

Icon information related to imaging findings is, for example, stored in advance in the medical information providing apparatus 2 and acquired by the display control unit 52 . The display control unit 52 causes the display position calculation unit 51 to set “REn” for the findings detected in the ultrasound CT image and “MGm” (n and m are 1 or more) for the image findings detected in the mammography image. (integer of ) and each identification number, and displayed as an icon in the schema image of each image.

Furthermore, by aligning the schema of the mammography image and the schema of the ultrasound CT image and comparing the image findings recorded in each schema, the type and size of the lesion and the overlooked lesion can be visualized. be able to. In this way, the properties of different modalities can be used to reduce false negatives and false positives.

In addition, by automatically creating a schema image when diagnosing the first image, it is possible to reduce the number of man-hours required for creating electronic medical charts, and reduce the burden on medical staff. In addition, when performing fully automated imaging with an ultrasonic CT imaging device, accuracy may not be guaranteed, so the results determined by the algorithm are presented, checked by technicians and doctors, and corrected if necessary. By saving, it is a big workability improvement compared to creating a table from scratch.

Of these multiple annotations, only the one with the highest possibility of malignancy may be superimposed on the first image, or all annotations may be superimposed and displayed. In addition, a notification may be issued to urge the attention of the medical staff as a region to be diagnosed more carefully with the first image by specifying the region including the periphery of a plurality of lesions as a region requiring precise interpretation.

Furthermore, it is possible to display a fine image by increasing the resolution of the area of the first image that requires precise interpretation. It is also possible to select the type of findings, such as displaying only tumors or including calcifications, on schemas and images. It is possible to operate in a way that reflects the thoughts of the person.

In addition, the mammography image, which is the second image, contains image information related to a region containing information other than a tumor such as calcification and tissue spasm, a mammary gland region, and a region suggesting the spread of a tumor. be. In that case, the biopsy region can be specified more accurately by detecting a single tumor or a plurality of tumors from that region with the ultrasonic CT imaging apparatus 1 .

In addition, it is known that invasive cancer, which requires treatment, has a high possibility of occurring in the mammary gland region and its margins among tissues in the breast. For this reason, the position display calculation unit 51 displays a guide of the boundary line of the mammary gland region detected in the mammography image on the ultrasound CT image or the schema, so that if the region suspected to be a tumor is outside the mammary gland region, False positives can be suspected. This is an effective means of reducing false positives by utilizing the fact that mammography provides high contrast between mammary glands and other tissues, whereas ultrasonic CT does not necessarily provide high contrast between mammary glands and other regions.

In addition, in the mammography image, which is the second image, the position information of individual tumors may be superimposed on the ultrasonic CT image and displayed. In ultrasound CT images, when the boundaries of image findings are unclear, by interspersing image finding information in mammography images, it is possible to estimate the range and boundaries of image findings in ultrasound CT images. become easier. In addition, image information may be included regarding regions suggestive of discrete extents in the mammography image. In that case, the biopsy region can be specified more accurately by detecting a single tumor or a plurality of tumors from that region with the ultrasonic CT imaging apparatus 1 .

Further, the description so far has been made on the premise that the second medical image diagnostic apparatus 2 is a mammography system. It is the same even if it is an ultrasonic imaging apparatus.

Note that when imaging is performed using the first medical image diagnostic apparatus 1 after imaging is performed using the second medical image diagnostic apparatus 2, the combination of these two types of modalities is as follows: The ultrasound CT imaging apparatus 1 is selected as the first medical image diagnostic apparatus 1, and mammography and other modalities can be selected as the second medical image diagnostic apparatus 2. FIG.

However, in view of the fact that the determination is made based on the image captured using the ultrasonic CT imaging apparatus 1, the ultrasonic CT imaging apparatus 1 is selected as the second medical image diagnostic apparatus 2, and the first medical imaging apparatus 1 is used. As the diagnostic imaging device 1, a handheld general-purpose ultrasonic diagnostic imaging device can also be selected. In this case, based on the interpretation report created based on the image taken by the ultrasonic CT imaging device 1, for example, a handheld general-purpose ultrasonic diagnostic imaging device for grasping the puncture position necessary for biopsy is assumed to be used.

Even in such a case, if necessary, the image acquired by the handheld general-purpose ultrasonic diagnostic imaging apparatus can be displayed overlapping the image captured by the ultrasonic CT imaging apparatus 1 again. , the puncture position can be grasped in more detail.

Further, each processing performed in the medical information providing apparatus 4 described above can also be performed in the ultrasonic CT imaging apparatus 1, for example. In this case, the ultrasonic CT imaging apparatus 1 includes the above-described display position calculation section and display control section.

The display position calculation unit 51 and the display control unit 52 have been described so far on the premise that they are processing units different from the control unit 49 . However, instead of such a configuration, for example, the control unit 49 may have a display position calculation function and a display control function, and the control unit 49 may execute processing according to each function.

For the display position calculation function of the display position calculation unit 51 or the control unit 49, programs such as a display position calculation program and a registration processing program stored in the storage unit 48 or the medical image storage system 3 are used. It can also be realized by having a processor execute it.

Here, the term "processor" in this specification refers to, for example, a dedicated or general-purpose CPU (Central Processing Unit), an arithmetic circuit (circuitry), or an Application Specific Integrated Circuit (ASIC), a programmable logic device (eg, Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)).

The processor realizes its functions by reading and executing a program stored in the storage unit 48 or the medical image storage system 3, or directly installed in the processor. A recording unit for storing programs may be provided individually for each processor, or the configuration of the storage unit 48 and the medical image storage system 3 as described above may be adopted. For the configuration of the storage unit, for example, a storage device such as a general RAM (Random Access Memory) such as a semiconductor or a magnetic disk or a HDD (Hard Disc Drive) is applied.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

1 first medical image diagnostic apparatus, ultrasonic CT imaging apparatus 2 second medical image diagnostic apparatus 3 medical image storage system 4 medical information providing apparatus 41 CPU
42 ROM
43 RAM
44... Input/output interface 45... Input unit 46... Display unit 47... Communication control unit 48... Storage unit 49... Control unit 51... Display position calculation unit 52... Display control unit 11 Bed 12 Measurement module 121 Bathtub 122 Ring array 123 Support member 124 Movement mechanism

Claims (13)

Acquiring first image information about a subject imaged by a first medical image diagnostic apparatus, and acquiring the subject imaged by a second medical image diagnostic apparatus prior to the first medical image diagnostic apparatus By acquiring second image information relating to and sharing the position information of the first image information and the second image information in the first image and the second image, a display position calculation unit that performs calculations so as to perform alignment processing with the second image;
a display control unit that reflects the position information of the second image calculated by the display position calculation unit on the display of the first image, and
A medical information providing device comprising: The display control unit reflects the position information of the second image on the display of the first image based on an instruction from a radiogram interpreter who uses the medical information providing apparatus to make a judgment on the subject. 2. The medical information providing apparatus according to claim 1, wherein the medical information providing apparatus wherein the subject is a breast,
3. The medical information providing apparatus according to claim 1, wherein said first medical image diagnostic apparatus is an ultrasonic CT imaging apparatus. 4. The second medical image diagnostic apparatus according to any one of claims 1 to 3, wherein the second medical image diagnostic apparatus is a mammographic X-ray apparatus, an ultrasonic CT imaging apparatus, or a general-purpose ultrasonic image diagnostic apparatus. The medical information providing device according to 1. 3. The second image is a two-dimensional image, the first image is a three-dimensional image, and the characteristic area of the second image is displayed in the first image. The medical information providing device according to any one of claims 4 6. The medical information providing apparatus according to any one of claims 1 to 5, wherein at least one of said first image and said second image further includes a schema indicating a part of the subject. 6. The display position calculation unit creates a schema indicating a part of the subject from at least one of the first image and the second image, and aligns the positions. A medical information providing device according to any one of the above. 7. The medical information providing apparatus according to claim 6, wherein said display position calculation unit further causes the interpretation report to display the characteristic region position in said schema. 8. The medical information providing apparatus according to any one of claims 1 to 7, wherein at least one of said first image and said second image further includes position information of a plurality of characteristic regions. Acquiring image information related to an image of a subject, acquiring position information related to a preceding image related to the subject that was previously captured, and acquiring position information of the image information of the image and the preceding image in the image and the preceding image a display position calculation unit that performs calculations so as to enable alignment processing between the image and the preceding image by sharing position information of image information of the images;
a display control unit that reflects the position information of the preceding image calculated by the display position calculation unit in the display of the image and displays the image;
An ultrasonic CT imaging apparatus comprising: a first medical image diagnostic apparatus that acquires first image information about a subject;
a second medical image diagnostic apparatus that acquires second image information regarding the subject prior to the first medical image diagnostic apparatus;
a medical information providing apparatus for providing medical information necessary for the determination to a radiologist who performs determination on the subject based on the first image information;
A medical information providing system comprising: 11. The medical information providing system according to claim 10, wherein said first medical image diagnostic apparatus is an ultrasonic CT imaging apparatus. 12. The method according to claim 10, wherein the medical information provided to the radiogram interpreter by the medical information providing apparatus is the first image information sharing the position information of the second image information. The medical information delivery system described.

Images