JP6858485B2 - Medical information processing device - Google Patents

Description

Embodiments of the present invention relate to a medical information processing device.

In breast cancer screening, a mammary gland image is imaged by a medical image diagnostic device such as a mammography device, an ultrasonic diagnostic device, or a magnetic resonance device. In breast cancer screening, not only the mammary gland image captured by a single medical image diagnostic device is read, but also the mammary gland image captured by a plurality of medical image diagnostic devices may be complementarily read for diagnosis. For example, interpretation is performed using a two-dimensional mammography image captured by a mammography apparatus and an ultrasonic image of a mammary gland captured by an ultrasonic diagnostic apparatus in combination.

Further, in recent years, in mammography apparatus, tomosynthesis imaging that generates a three-dimensional image by changing the X-ray irradiation angle with respect to the breast of the subject while fixing the breast of the subject has become widespread. A three-dimensional mammography image generated by tomosynthesis imaging is called a tomosynthesis image. In the following, the two-dimensional mammography image and the tomosynthesis image will be collectively referred to as a mammography image.

JP-A-2015-207450

An object to be solved by the present invention is to provide a medical information processing apparatus capable of streamlining a workflow in an examination in which a tomosynthesis image and an ultrasonic image are used in combination.

The medical information processing apparatus of the embodiment includes a first storage unit, a second storage unit, a calculation unit, and an image generation processing unit. The first storage unit stores the position of the region of interest definitive in the schematic diagram of the breast position of the region of interest at the time of scanning the breast of a subject using an ultrasonic probe associated. The second storage unit stores a tomosynthesis image , which is a three-dimensional image of the subject acquired by the mammography apparatus. The calculation unit obtains the position on the tomosynthesis image based on the position of the region of interest attached to the schematic diagram of the breast. The image generation processing unit generates an output image based on the position on the tomosynthesis image.

FIG. 1 is a diagram showing a configuration example of a medical information processing device according to the first embodiment. FIG. 2 is a diagram (1) showing a configuration example of the mammography apparatus according to the first embodiment. FIG. 3 is a diagram (2) showing a configuration example of the mammography apparatus according to the first embodiment. FIG. 4 is a diagram showing a configuration example of the ultrasonic diagnostic apparatus according to the first embodiment. FIG. 5 is a diagram showing an example of a body mark according to the first embodiment. FIG. 6 is a diagram showing an example of mammary gland image diagnosis using a combination of tomosynthesis image and ultrasonic image. FIG. 7 is a diagram showing a configuration example of the image processing apparatus according to the first embodiment. FIG. 8 is a flowchart (1) showing a processing procedure by the image processing apparatus according to the first embodiment. FIG. 9A is a diagram (1) for explaining the first embodiment. FIG. 9B is a diagram (2) for explaining the first embodiment. FIG. 9C is a diagram (3) for explaining the first embodiment. FIG. 10 is a flowchart (2) showing a processing procedure by the image processing apparatus according to the first embodiment. FIG. 11A is a diagram (4) for explaining the first embodiment. FIG. 11B is a diagram (5) for explaining the first embodiment. FIG. 12A is a diagram (6) for explaining the first embodiment. FIG. 12B is a diagram (7) for explaining the first embodiment. FIG. 13 is a diagram (8) for explaining the first embodiment. FIG. 14 is a diagram (9) for explaining the first embodiment. FIG. 15 is a diagram showing a configuration example of the image processing apparatus according to the second embodiment. FIG. 16 is a flowchart (1) showing a processing procedure by the image processing apparatus according to the second embodiment. FIG. 17A is a diagram (1) for explaining the second embodiment. FIG. 17B is a diagram (2) for explaining the second embodiment. FIG. 17C is a diagram (3) for explaining the second embodiment. FIG. 18 is a flowchart (2) showing a processing procedure by the image processing apparatus according to the second embodiment. FIG. 19 is a diagram (4) for explaining the second embodiment.

Hereinafter, embodiments of the medical information processing apparatus will be described with reference to the drawings.

(First Embodiment)
FIG. 1 is a diagram showing a configuration example of the medical information processing apparatus 100 according to the first embodiment. The medical information processing apparatus 100 according to the present embodiment is installed in a hospital where breast cancer screening is performed, and is used for breast image diagnosis using a mammography image and an ultrasonic image in combination. For example, as shown in FIG. 1, the medical information processing apparatus 100 according to the present embodiment includes a mammography apparatus 10, an ultrasonic diagnostic apparatus 20, an image processing apparatus 30, and an image display apparatus 40. The devices are connected to each other via the network 50, and transmit and receive images captured by the mammography device 10 and the ultrasonic diagnostic device 20 to and from each other. For example, when a PACS (Picture Archiving and Communication System) is introduced in the medical information processing apparatus 100, each apparatus is a DICOM (Digital Imaging and Communications in Medicine) format medical image data in which incidental information is added to the medical image data. Etc. are sent and received to each other. Here, the incidental information includes, for example, a patient ID (Identifier) for identifying a patient, an examination ID for identifying an examination, a device ID for identifying each device, a series ID for identifying one imaging by each device, and the like. Is done. In the embodiment shown below, a case where the medical information processing device 100 is realized by having a mammography device 10, an ultrasonic diagnostic device 20, an image processing device 30, and an image display device 40 will be described. The medical information processing device according to the present invention may be realized by using any one of a mammography device 10, an ultrasonic diagnostic device 20, an image processing device 30, and an image display device 40. Alternatively, the medical information processing device according to the present invention may be realized by arbitrarily combining any one of the mammography device 10, the ultrasonic diagnostic device 20, the image processing device 30, and the image display device 40. That is, the medical information processing device according to the present invention can be realized by a single device, or by arbitrarily combining a plurality of devices.

The mammography apparatus 10 irradiates the breast of the subject with X-rays, detects the X-rays that have passed through the breast, and generates a mammography image.

2 and 3 are diagrams showing a configuration example of the mammography apparatus 10 according to the first embodiment. For example, as shown in FIG. 2, the mammography apparatus 10 has a base 11 and a stand 12. The stand 12 is erected on the base 11, and supports the photographing table 13, the compression plate 14, the X-ray output device 15, and the X-ray detection device 16. The photographing table 13, the compression plate 14, and the X-ray detection device 16 are supported so as to be movable in the vertical direction.

The imaging table 13 is a table that supports the breast B of the subject, and has a support surface 13a on which the breast B is placed. The compression plate 14 is arranged above the shooting table 13 and is provided so as to face the shooting table 13 in parallel and move in a direction in which the compression plate 14 is in contact with and separated from the shooting table 13. The compression plate 14 presses the breast B supported on the imaging table 13 when it moves in the direction approaching the imaging table 13. The breast B compressed by the compression plate 14 is spread thinly, and the overlap of the mammary glands in the breast B is reduced.

Further, as shown in FIG. 3, the mammography apparatus 10 controls communication with an input interface 17a, an elevating drive circuit 17b, a high voltage generator 17c, an image processing circuit 17d, an image storage circuit 17e, and a display 17f. It has an interface 17g and a system control circuit 17h. The input interface 17a receives input operations of various commands from the operator. The elevating drive circuit 17b is connected to the photographing table 13 and raises and lowers the photographing table 13 in the vertical direction. Further, the elevating drive circuit 17b is connected to the compression plate 14 and elevates the compression plate 14 in the vertical direction (direction in which the compression plate 14 is brought into contact with the image pickup table 13).

The X-ray output device 15 includes an X-ray tube 15a and an X-ray squeezer 15b. The X-ray tube 15a generates X-rays. The X-ray diaphragm 15b is arranged between the X-ray tube 15a and the compression plate 14, and controls the irradiation range of X-rays generated from the X-ray tube 15a. The high voltage generator 17c is connected to the X-ray tube 15a and supplies a high voltage for the X-ray tube 15a to generate X-rays.

The X-ray output device 15 is capable of tomosynthesis imaging. In tomosynthesis imaging, the positions of the imaging table 13 and the compression plate 14 are fixed, and X-rays are output by changing the angle of the X-ray tube 15a with respect to the breast while compressing the breast of the subject.

The X-ray detection device 16 includes an X-ray detector 16a and a signal processing circuit 16b. The X-ray detector 16a detects the X-rays transmitted through the breast B and the imaging table 13 and converts them into an electric signal (transmitted X-ray data). The signal processing circuit 16b generates X-ray projection data from the electric signal converted by the X-ray detector 16a.

The image processing circuit 17d is connected to the signal processing circuit 16b and the image storage circuit 17e, and generates a mammography image based on the X-ray projection data generated by the signal processing circuit 16b. Here, the image processing circuit 17d is generated by, for example, fixing the positions of the imaging table 13 and the compression plate 14 in the MLO (Mediolateral-Oblique) direction and outputting X-rays without changing the angle with respect to the breast. A mammography image (MLO image) is generated based on the X-ray projection data. Further, the image processing circuit 17d was generated by, for example, fixing the positions of the imaging table 13 and the compression plate 14 in the CC (Cranio-Caudal) direction and outputting X-rays without changing the angle with respect to the breast. A mammography image (CC image) is generated based on the X-ray projection data.

Further, the image processing circuit 17d generates a tomosynthesis image, which is a three-dimensional image, based on images taken from each of a plurality of angles with respect to the subject. Here, the image processing circuit 17d, for example, fixes the positions of the imaging table 13 and the compression plate 14 in the MLO direction, changes the angle with respect to the breast, outputs X-rays, and outputs X-rays based on the generated X-ray projection data. Generate a tomosynthesis image (MLO tomosynthesis image). Further, the image processing circuit 17d, for example, fixes the positions of the imaging table 13 and the compression plate 14 in the CC direction, changes the angle with respect to the breast, outputs X-rays, and outputs tomosynthesis based on the generated X-ray projection data. Generate an image (CC tomosynthesis image).

Specifically, the image processing circuit 17d generates a tomosynthesis image by a predetermined process based on a plurality of images corresponding to a plurality of angles with respect to the subject. The predetermined processing is, for example, a shift addition method, a filtered back projection (FBP) method, or the like. That is, by tomosynthesis imaging, a three-dimensional image is generated by irradiating X-rays from the X-ray tube 15a and imaging the breast of the subject from different directions. In the following, when describing a mammography image, in addition to the MLO image and the CC image, the MLO tomosynthesis image and the CC tomosynthesis image are also included. Further, in order to distinguish it from the tomosynthesis image, the MLO image and the CC image are referred to as a two-dimensional mammography image. For example, an MLO image is called a two-dimensional MLO image, and a CC image is called a two-dimensional CC image.

The image processing circuit 17d stores the generated mammography image in the image storage circuit 17e. Further, the image processing circuit 17d is connected to the display 17f and displays the generated mammography image on the display 17f. The image processing circuit 17d can switch the type of mammography image to be created based on the input operation from the input interface 17a. In addition to the mammography image generated by the image processing circuit 17d, the image storage circuit 17e may store, for example, image data generated by the ultrasonic diagnostic apparatus 20 or the image processing apparatus 30.

The communication control interface 17g controls the communication performed with other devices via the network 50. For example, the communication control interface 17g transfers the mammography image generated by the image processing circuit 17d to another device via the network 50. The mammography image transferred via the network 50 can be image-displayed or image-processed in the transfer-destination device.

The system control circuit 17h is connected to an input interface 17a, an elevating drive circuit 17b, a high voltage generator 17c, an X-ray filter 15b, an image processing circuit 17d, and a communication control interface 17g, and is connected to the mammography apparatus 10. Control the whole as a whole.

Returning to FIG. 1, the ultrasonic diagnostic apparatus 20 generates an ultrasonic image based on the reflected wave data collected by scanning the subject with an ultrasonic probe that transmits and receives ultrasonic waves.

FIG. 4 is a diagram showing a configuration example of the ultrasonic diagnostic apparatus 20 according to the first embodiment. As shown in FIG. 4, the ultrasonic diagnostic apparatus 20 according to the present embodiment includes an ultrasonic probe 21, an input interface 22, a display 23, and an apparatus main body 24. The ultrasonic probe 21 is communicably connected to the transmission / reception circuit 24a included in the apparatus main body 24 described later. Further, the input interface 22 and the display 23 are communicably connected to various circuits included in the apparatus main body 24.

The ultrasonic probe 21 comes into contact with the body surface of the subject P to transmit and receive ultrasonic waves. For example, the ultrasonic probe 21 has a plurality of piezoelectric vibrators (also referred to as oscillators). These plurality of piezoelectric vibrators generate ultrasonic waves based on the transmission signal supplied from the transmission / reception circuit 24a. The generated ultrasonic waves are reflected in the body tissue of the subject P and are received by a plurality of piezoelectric vibrators as reflected wave signals. The ultrasonic probe 21 sends the reflected wave signal received by the plurality of piezoelectric vibrators to the transmission / reception circuit 24a.

In the first embodiment, even if the ultrasonic probe 21 is a 1D array probe that scans a two-dimensional region in the subject P (two-dimensional scanning), the ultrasonic probe 21 scans the three-dimensional region in the subject P (2D scanning). It can also be applied to a mechanical 4D probe or a 2D array probe that performs three-dimensional scanning).

The input interface 22 corresponds to, for example, a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, a trackball, a joystick, and the like. The input interface 22 receives various setting requests from the operator of the ultrasonic diagnostic apparatus 20, and appropriately transfers the accepted various setting requests to each circuit of the apparatus main body 24.

The display 23 displays a GUI (Graphical User Interface) for the operator to input various setting requests using the input interface 22, and an image (ultrasonic image) based on the ultrasonic image data generated in the apparatus main body 24. ) Etc. are displayed.

The device main body 24 is a device that generates ultrasonic image data based on the reflected wave signal received by the ultrasonic probe 21. As shown in FIG. 4, the apparatus main body 24 includes, for example, a transmission / reception circuit 24a, a B mode processing circuit 24b, a Doppler processing circuit 24c, an image generation circuit 24d, an image memory 24e, a storage circuit 24f, and a processing circuit. It has 24 hours. The transmission / reception circuit 24a, the B-mode processing circuit 24b, the Doppler processing circuit 24c, the image generation circuit 24d, the image memory 24e, the storage circuit 24f, the communication control interface 24g, and the processing circuit 24h are communicably connected to each other.

The transmission / reception circuit 24a controls the transmission / reception of ultrasonic waves by the ultrasonic probe 11. For example, the transmission / reception circuit 24a controls the ultrasonic wave transmission / reception performed by the ultrasonic probe 21 based on the instruction of the processing circuit 24h described later. The transmission / reception circuit 24a creates transmission waveform data, and generates a transmission signal for the ultrasonic probe 21 to transmit ultrasonic waves from the created transmission waveform data. Then, the transmission / reception circuit 24a applies a transmission signal to the ultrasonic probe 21 to transmit an ultrasonic beam in which ultrasonic waves are focused in a beam shape.

Further, in the transmission / reception circuit 24a, the reflected wave signal received by the ultrasonic probe 21 is given a predetermined delay time to perform addition processing, so that the reflected component is emphasized from the direction corresponding to the reception directivity of the reflected wave signal. The reflected wave data is generated, and the generated reflected wave data is transmitted to the B mode processing circuit 24b and the Doppler processing circuit 24c.

For example, the transmission / reception circuit 24a includes an amplifier circuit (appropriately described as "Amp"), an A / D (Analog / Digital) converter (appropriately described as "ADC"), a generation circuit, and an orthogonal detection circuit (appropriately described as "IQ"). It is described as) and so on. The amplifier circuit amplifies the reflected wave signal for each channel and performs gain correction processing. The A / D converter A / D-converts the gain-corrected reflected wave signal.

The generation circuit provides the digital data with the reception delay time required to determine the reception directivity. Then, the generation circuit performs addition processing of the reflected wave signal given the reception delay time. The addition process of the generation circuit emphasizes the reflected component from the direction corresponding to the reception directivity of the reflected wave signal.

Then, the orthogonal detection circuit converts the output signal of the adder into an in-phase signal (I signal, I: In-phase) and an orthogonal signal (Q signal, Q: Quadrature-phase) in the baseband band. Then, the orthogonal detection circuit stores the I signal and the Q signal (hereinafter, referred to as IQ signal) as reflected wave data in the buffer. The orthogonal detection circuit may convert the output signal of the adder into an RF (Radio Frequency) signal and then store it in the buffer. The IQ signal and the RF signal are signals (received signals) including phase information. Although the orthogonal detection circuit has been described as being arranged after the generation circuit, the embodiment is not limited to this. For example, the orthogonal detection circuit may be arranged in front of the generation circuit. In such a case, the generation circuit performs an addition process of the I signal and the Q signal.

The B-mode processing circuit 24b performs various signal processing on the reflected wave data generated by the transmission / reception circuit 24a from the reflected wave signal. The B-mode processing circuit 24b performs logarithmic amplification, envelope detection processing, etc. on the reflected wave data received from the transmission / reception circuit 24a, and the signal strength for each sample point (observation point) is expressed by the brightness of the brightness. Data (B mode data) is generated. The B-mode processing circuit 24b sends the generated B-mode data to the image generation circuit 24d.

The Doppler processing circuit 24c generates data (Doppler data) obtained by extracting motion information based on the Doppler effect of a moving body at each sample point in the scanning region from the reflected wave data received from the transmission / reception circuit 24a. Specifically, the Doppler processing circuit 24c generates Doppler data obtained by extracting the average velocity, dispersion value, power value, etc. at each sample point as motion information of the moving body. Here, the moving body is, for example, blood flow, tissues such as the heart wall, and a contrast medium. The Doppler processing circuit 24c sends the generated Doppler data to the image generation circuit 24d.

The image generation circuit 24d generates ultrasonic image data from the data generated by the B mode processing circuit 24b and the Doppler processing circuit 24c. For example, the image generation circuit 24d generates B-mode image data in which the intensity of the reflected wave is represented by the brightness from the B-mode data generated by the B-mode processing circuit 24b. Further, the image generation circuit 24d generates Doppler image data representing mobile information from the Doppler data generated by the Doppler processing circuit 24c. This Doppler image data is speed image data, distributed image data, power image data, or image data in which these are combined.

The image memory 24e is a memory that stores data generated by the B-mode processing circuit 24b, the Doppler processing circuit 24c, and the image generation circuit 24d. For example, after the diagnosis, the operator can recall the image recorded during the examination, and it is possible to reproduce the image as a still image or as a moving image using a plurality of images. The image memory 24e may store an image luminance signal after passing through the transmission / reception circuit 24a, other raw data, image data acquired via the network 50, and the like. For example, the image memory 24e may store image data or the like generated by the mammography apparatus 10 or the image processing apparatus 30.

The storage circuit 24f stores various data such as a device control program for performing ultrasonic transmission / reception, image processing, and display processing, diagnostic information (for example, patient ID, doctor's findings, etc.), diagnostic protocol, and various setting information. To do. The storage circuit 24f may be used for storing an image stored in the image memory 24e. Further, the data stored in the storage circuit 24f can be transferred to an external device via an interface (not shown).

Further, the storage circuit 24f stores the body mark. FIG. 5 is a diagram showing an example of a body mark according to the first embodiment. The example shown in FIG. 5 shows a schematic diagram of the mammary gland region as an example of a body mark schematically representing the breast. For example, as shown in FIG. 5, in the schematic diagram of the mammary gland region, for each of the left and right breasts, a circular region representing the breast region (hereinafter referred to as the breast region) and a substantially triangular region representing the axillary region (hereinafter referred to as the axillary region). , Axillary region).

Here, the circular region representing the breast region is divided into upper and lower regions and left and right regions, and is divided into four regions "A" to "D". For example, the region "A" (hereinafter, region A) indicates the region of the upper medial side of the breast, and the region "B" (hereinafter, region B) indicates the region of the lower medial side of the breast. Further, for example, the “C” region (hereinafter, C region) indicates the region of the outer upper part of the breast, and the “D” region (hereinafter, the D region) indicates the region of the outer lower part of the breast. Further, the substantially triangular region "C'" (hereinafter referred to as "C'region") representing the axillary region has a shape that extends diagonally upward from the region C and becomes thinner as the distance from the region C increases. Further, for example, the region of "E" (hereinafter referred to as "E region") (hereinafter referred to as "E region"), which is not shown in FIG. 5, indicates the areola portion. As the schematic diagram referred to here, various diagrams can be used as long as they show the positional relationship in the breast.

The communication control interface 24g controls the communication performed with other devices via the network 50. For example, the communication control interface 24g transfers the ultrasonic image generated by the image generation circuit 24d to another device via the network 50. The ultrasonic image transferred via the network 50 can be image-displayed or image-processed in the transfer-destination device.

The processing circuit 24h controls the entire processing of the ultrasonic diagnostic apparatus 20. Specifically, the processing circuit 24h processes the transmission / reception circuits 24a and B mode based on various setting requests input from the operator via the input interface 22, various control programs read from the storage circuit 24f, and various data. It controls the processing of the circuit 24b, the Doppler processing circuit 24c, the image generation circuit 24d, and the like. Further, the processing circuit 24h causes the display 23 to display the ultrasonic image data stored in the image memory 24e.

In addition, the processing circuit 24h generates scanning position information in which the position of the region of interest when the breast of the subject P is scanned using the ultrasonic probe 21 is associated with the schematic diagram of the breast. For example, the processing circuit 24h generates scanning position information in which a probe mark schematically showing an ultrasonic probe 21 is superimposed on a position corresponding to a region of interest during ultrasonic scanning on a schematic diagram.

More specifically, the processing circuit 24h receives the arrangement of the probe mark on the schematic diagram from the operator via the input interface 22. Here, the operator places the probe mark at a position corresponding to the region of interest during ultrasonic scanning on the schematic diagram. As a result, the processing circuit 24h generates scanning position information. The processing circuit 24h stores the generated scanning position information in the storage circuit 24f. The processing circuit 24h stores the identification information (patient ID) that can uniquely identify the subject in the storage circuit 24f in association with the scanning position information. In other words, the storage circuit 24f stores the identification information that can uniquely identify the subject in association with the scanning position information in which the position of the region of interest is attached to the schematic diagram of the breast. Further, the processing circuit 24h stores the scanning position information in the storage circuit 24f in the DICOM format or the JPEG format. In other words, the storage circuit 24f stores the scanning position information in DICOM format or JPEG format.

Returning to FIG. 1, the image processing device 30 processes a mammography image generated by the mammography device 10, an ultrasonic image generated by the ultrasonic diagnostic device 20, and the like. The image processing device 30 is mainly used when a mammography inspection is performed by a mammography inspection engineer. Further, the image processing device 30 receives input of findings related to the mammography image from a mammography inspection engineer or a radiologist, and stores information indicating the accepted findings as finding information. For example, the image processing device 30 is an image storage server, a workstation, or the like.

The image display device 40 acquires and displays a mammography image, an ultrasonic image, finding information on the mammography image, and the like from the image processing device 30. The image display device 40 is mainly used when an ultrasonic examination is performed by an ultrasonic examination engineer or a radiologist. For example, the image display device 40 is a tablet terminal that can be carried by an operator and can be connected to the network 50 via a wireless LAN (Local Area Network). The image display device 40 may be, for example, a notebook personal computer or a desktop personal computer.

In such a medical information processing apparatus 100, a mammary gland image diagnosis using a combination of a tomosynthesis image and an ultrasonic image is performed in a breast cancer screening. FIG. 6 is a diagram showing an example of mammary gland image diagnosis using a combination of tomosynthesis image and ultrasonic image.

Conventionally, when performing an ultrasonic examination with reference to a two-dimensional mammography (2DMG) image, the ultrasonic inspector understands the imaging position of the ultrasonic image, and the region of interest of the ultrasonic image and 2 Grasp the correspondence with the dimensional mammography image (see the left figure in FIG. 6). Here, the region of interest is, for example, a region determined to be a lesion site on an ultrasonic image. Therefore, a more detailed diagnosis may be performed on the region of interest by using an ultrasonic image and a mammography image in combination. Further, when the ultrasonic examination is performed while referring to the tomosynthesis image, the ultrasonic inspector also needs to grasp the correspondence relationship between the region of interest of the ultrasonic image and the tomosynthesis image (see the right figure of FIG. 6). ).

Here, the two-dimensional mammography image or tomosynthesis image compresses the breast and images the breast. In addition, the subject is in the supine position during the ultrasonic examination. That is, the morphology of the breast is different between the two-dimensional mammography image or tomosynthesis image and the ultrasonic image. Therefore, it is not easy to grasp the correspondence between the region of interest of the ultrasonic image and the two-dimensional mammography image or the tomosynthesis image.

Here, a technique is disclosed in which the correspondence relationship with the imaging position by the ultrasonic diagnostic apparatus is shown on a schematic diagram of the breast of the ultrasonic diagnostic apparatus by using the MLO image and the CC image of the two-dimensional mammography image. However, this technique associates the corresponding ultrasonic imaging region based on the information of the mammography image, and does not associate the imaging position by the ultrasonic diagnostic apparatus with the mammography image or tomosynthesis image.

Furthermore, in the tomosynthesis image, due to the reconstruction algorithm, in a very small object such as microcalcification, if the object does not exist in the reconstructed cross section, the object may be blurred or the object may not be reflected. Sometimes. In addition, in order to make a better diagnosis based on tomosynthesis images, it is important to avoid overlapping of areas of interest and areas of interest and mammary gland tissue, and to set the area of interest in an appropriate cross section.

For this reason, the image processing apparatus 30 according to the first embodiment sets the region of interest by the ultrasonic diagnostic apparatus as an MLO tomosynthesis image or a CC tomosynthesis image in a diagnosis using images of breasts having different shapes of objects. Correspond to. The details of the image processing apparatus 30 will be described below.

FIG. 7 is a diagram showing a configuration example of the image processing device 30 according to the first embodiment. As shown in FIG. 7, the image processing device 30 includes an input interface 31, a display 32, a communication control interface 33, a storage circuit 34, and a processing circuit 35.

The input interface 31 receives various operations and various information inputs from the operator. For example, the input interface 31 is a keyboard, mouse, button, trackball, touch panel, or the like.

The display 32 displays a GUI and various images for receiving various operations from the operator. For example, the display 32 is a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, or the like.

The communication control interface 33 controls the communication performed with other devices via the network 50. For example, the communication control interface 33 is a network card or a network adapter, and communicates with other devices by connecting to the network 50 via a LAN of Ethernet (registered trademark). Further, for example, the communication control interface 33 connects to the network 50 via a wireless LAN to perform wireless communication with other devices.

The storage circuit 34 is a storage device that stores various data such as image data 34a, finding information 34b, and patient information 34c, and a control program 34d for performing image processing and display processing. Further, the data stored in the storage circuit 34 can be transferred to an external device via an interface (not shown). Further, the storage circuit 34 may store image data and the like generated by the mammography apparatus 10 and the ultrasonic diagnostic apparatus 20.

Various data stored in the storage circuit 34 will be described. For example, the storage circuit 34 stores, as image data 34a, a mammography image of the breast of the subject and information indicating the imaging direction of the mammography image. Specifically, the storage circuit 34 stores the mammography image and the information indicating the photographing direction in association with each image. The storage circuit 34 stores a mammography image and information indicating a shooting direction by an image data acquisition function 35a described later.

More specifically, the storage circuit 34 stores the MLO image and the CC image. Further, the storage circuit 34 stores an MLO tomosynthesis image and a CC tomosynthesis image. The information indicating the shooting direction referred to here is, for example, position information represented by the device coordinate system of the mammography apparatus, and is added to each image as incidental information when the mammography image is generated by the mammography apparatus.

Further, for example, the storage circuit 34 stores the finding information 34b regarding the mammography image of the subject. The finding information is stored in the storage circuit 34 by the finding information creating function 35b described later.

Further, for example, the storage circuit 34 stores the patient information 34c that can uniquely identify the patient. For example, the patient information includes a patient ID (Identifier), a name, an age, a medical examination history, and the like. Further, for example, the storage circuit 34 stores the control program 34d. The control program 34d includes a program corresponding to each function. This control program 34d is read by the processing circuit 35. Then, the processing circuit 35 realizes a function corresponding to each program by executing the control program 34d read from the storage circuit 34.

The processing circuit 35 controls the operation of the image processing device 30. As shown in FIG. 7, the processing circuit 35 includes an image data acquisition function 35a, a finding information creation function 35b, a display control function 35c, a position identification function 35d, a position information generation function 35e, and a transmission function 35f. Execute. Here, for example, the image data acquisition function 35a, which is a component of the processing circuit 35 shown in FIG. 7, the finding information creation function 35b, the display control function 35c, the position identification function 35d, and the position information generation function 35e. Each processing function executed by the transmission function 35f is recorded in the storage circuit 34 in the form of a program that can be executed by a computer. The processing circuit 35 is a processor that realizes a function corresponding to each program by reading each program from the storage circuit 34 and executing the program. In other words, the processing circuit 35 in the state where each program is read has each function shown in the processing circuit 35 of FIG.

The image data acquisition function 35a acquires a mammography image of the breast of the subject and information indicating the imaging direction of the mammography image. Specifically, the image data acquisition function 35a communicates with the mammography apparatus 20 via the communication control interface 33 to obtain a mammography image of the subject to be diagnosed and information indicating the shooting direction of the mammography image. The acquired mammography image and information indicating the shooting direction are stored in the storage circuit 34. Here, the image data acquisition function 35a acquires at least one of an MLO tomosynthesis image and a CC tomosynthesis image for either the left or right breast of the subject.

Further, the image data acquisition function 35a acquires scanning position information from the storage circuit 24f of the ultrasonic diagnostic apparatus 20. Then, the image data acquisition function 35a stores the identification information that can uniquely identify the subject in the storage circuit 34 in association with the acquired scanning position information.

The finding information creating function 35b creates finding information regarding the mammography image of the subject based on the finding input from the operator. Specifically, the finding information creation function 35b receives input of findings related to a mammography image from a mammography inspection engineer via the input interface 31. Then, the finding information creation function 35b creates finding information indicating the received finding. The finding information creating function 35b stores the created finding information in the storage circuit 34.

The display control function 35c displays a reference screen for referencing the mammography image on the display 32. Specifically, when the display control function 35c receives a display request from the operator via the input interface 31, the display control function 35c reads a mammography image of the subject to be diagnosed from the storage circuit 34 and finds the subject to be diagnosed. Information is read from the storage circuit 34b. Then, the display control function 35c displays the reference screen on which the read mammography image and the finding information are arranged on the display 32. Further, the display control function 35c displays an output image described later on the display 32. The output image is also referred to as position identification information.

The position specifying function 35d is an example of a calculation unit. The position identification function 35d obtains the position on the three-dimensional image based on the position of the region of interest in the ultrasonic examination attached to the schematic diagram of the breast. For example, the position specifying function 35d obtains the position information of the region of interest on the three-dimensional image from the scanning position information. Here, the position specifying function 35d acquires the scanning position information corresponding to the designated identification information from the storage circuit 34, and obtains the position information of the region of interest on the three-dimensional image from the scanning position information. The position specifying function 35d may directly acquire scanning position information from the storage circuit 24f of the ultrasonic diagnostic apparatus 20.

The position information generation function 35e is an example of an image generation processing unit. The position information generation function 35e generates an output image based on the position on the three-dimensional image. For example, the position information generation function 35e generates an output image in which the position information of the region of interest is associated with the three-dimensional image. Then, the position information generation function 35e stores the output image and the scanning position information used when generating the output image in the storage circuit 34 in association with each other.

Subsequently, the processing operations of the position specifying function 35d and the position information generating function 35e will be described with reference to FIGS. 8 to 10. FIG. 8 is a flowchart showing a processing procedure by the image processing apparatus 30 according to the first embodiment, and FIGS. 9A to 9C are diagrams for explaining the first embodiment. FIG. 10 is a flowchart showing a processing procedure by the image processing apparatus 30 according to the first embodiment. Further, FIGS. 8 and 9A to 9C describe a case where a CC tomosynthesis image is used, and FIG. 10 describes a case where an MLO tomosynthesis image is used. When the CC tomosynthesis image is used and when the MLO tomosynthesis image is used, the position identification function 35d specifies the relative position of the region of interest in the schematic diagram of the breast and obtains the position information of the region of interest on the three-dimensional image. .. Then, the position information generation function 35e generates an output image in which the position information of the region of interest is associated with the three-dimensional image.

First, a case where a CC tomosynthesis image is used will be described. As shown in FIG. 8, the position specifying function 35d obtains the intersection of the line L1 extending in the body axis direction passing through the region of interest and the portion corresponding to the circle in the schematic diagram of the breast, and obtains the ratio of the positions of the regions of interest in L1. (Step S1). In other words, the position specifying function 35d determines the relative position of the region of interest using the intersection of the first straight line passing through the region of interest and parallel to the body axis direction in the schematic diagram and the schematic diagram. For example, when the scanning position information shown in FIG. 9A is acquired, the position specifying function 35d obtains (a: b) as the ratio of the positions of the regions of interest in L1. In FIG. 9A, a linear probe mark is shown on the schematic diagram of the right breast. In the example shown in FIG. 9A, the schematic diagram is a circle having a radius X, and the papilla position is the origin. Further, in the example shown in FIG. 9A, a cross mark attached substantially in the center of the probe mark indicates a point P which is a region of interest. In the example of FIG. 9A, the position of the region of interest is substantially the center of the probe mark, but the embodiment is not limited to this. For example, the location of the region of interest may be indicated at one end of the probe mark.

Then, the position specifying function 35d obtains the intersection of the lines L2 and L1 passing through the nipple and perpendicular to the body axis and the intersection of L2 and the portion corresponding to the circle in the schematic diagram, and obtains the ratio of the positions of the regions of interest in L2 (step S2). ). In other words, the position identification function 35d determines the relative position of the region of interest using the intersection of the second straight line passing through the papilla and perpendicular to the body axis direction and the first straight line in the schematic diagram. For example, the position specifying function 35d obtains (c: d) as the ratio of the positions of the regions of interest in L2, as shown in FIG. 9A. Location capability 35d are the coordinates of the point P (-d, e) if a, c = X-d, e = √ (X 2 -d 2) (b-a / a + b) of the point P with Calculate the coordinates.

Subsequently, the position specifying function 35d determines the corresponding cross section using the ratio (a: b) between the cross section closest to the compression plate 14 and the cross section closest to the X-ray detector 16a (step S3). In other words, the position identification function 35d identifies the relative position of the cross section including the region of interest from the three-dimensional image based on the relative position of the determined region of interest. In the example shown in FIG. 9B, a case where the breast of the subject is compressed and fixed between the compression plate 14 and the X-ray detector 16a is shown. Each cross section of the tomosynthesis image generated when tomosynthesis imaging is performed in this state is shown by a broken line. As shown in FIG. 9B, in the position identification function 35d, the ratio of the intersection of the straight line connecting the compression plate 14 and the X-ray detector 16a with the shortest distance and the cross section including the region of interest of the tomosynthesis image is (a: b). Determine the cross section to be. That is, the position specifying function 35d determines that the region of interest is included in the second cross section from the compression plate 14.

Further, as shown in FIG. 9C, the position specifying function 35d calculates the papilla and the breast start (end) position in the tomosynthesis image, determines the position to be the ratio (c: d), and on the CC tomosynthesis image. The corresponding position is determined (step S4). In other words, the position specifying function 35d obtains the position information of the region of interest in the cross section including the region of interest based on the relative position of the determined region of interest. In the example of FIG. 9C, it is shown that the region of interest is on the line where the ratio is c: d on the CC tomosynthesis image of the cross section determined in step S3. Then, the position information generation function 35e generates position identification information by showing a line at the position determined in step S4 on the CC tomosynthesis image of the cross section determined in step S3. The position information generation function 35e may generate position identification information including the ratio (c: d) shown in FIG. 9C, for example. Further, the position information generation function 35e may use only the CC tomosynthesis image of the cross section determined in step S3 as the position identification information.

Next, a case where an MLO tomosynthesis image is used will be described. As shown in FIG. 10, the position identification function 35d finds the intersection of the line L1 extending in the direction of the X-ray tube 15a passing through the region of interest and the portion corresponding to the circle in the schematic diagram of the breast, and the ratio of the positions of the regions of interest in L1. (A: b) is obtained (step S11). Then, the position specifying function 35d finds the intersection of the lines L2 and L1 passing through the nipple and perpendicular to the body axis, and finds the ratio (c: d) of the positions of the regions of interest in L2 (step S12).

Subsequently, the position information generation function 35e determines the corresponding cross section using the ratio (a: b) between the cross section closest to the compression plate 14 and the cross section closest to the X-ray detector 16a (step S13). Then, the position information generation function 35e calculates the papilla and the breast start (end) position in the tomosynthesis image, determines the position to be the ratio (c: d), and determines the corresponding position on the LMO tomosynthesis image. (Step S14). As described above, the position information generation function 35e generates position identification information by indicating the position determined in step S14 on the MLO tomosynthesis image of the cross section determined in step S13.

In the above-described embodiment, the processing circuit 24h has been described as accepting the arrangement of the probe mark on the schematic diagram from the operator via the input interface 22, but the embodiment is not limited to this. Absent. For example, when the ultrasonic probe 21 has a position sensor, the processing circuit 24h of the ultrasonic diagnostic apparatus 20 is interested in the schematic diagram of the breast based on the position information measured by the position sensor of the ultrasonic probe 21. The position of the region may be obtained. Here, when the ultrasonic probe 21 has a position sensor, the processing circuit 24h generates a schematic diagram of the breast by further determining a landmark position based on the position information measured by the position sensor. Hereinafter, a process of generating position identification information when the ultrasonic probe 21 has a position sensor will be described with reference to FIGS. 11A and 11B and FIGS. 12A and 12B. 11A to 12B are diagrams for explaining the first embodiment.

In FIGS. 11A and 11B, the case where the axilla, the papilla, and the rib body located parallel to the papilla are used as landmarks will be described. As shown in FIG. 11A, the operator of the ultrasonic diagnostic apparatus 20 abuts the ultrasonic probe 21 on the axillary fossa and measures the position with the position sensor, and then abuts the ultrasonic probe 21 on the papilla and positions the position with the position sensor. To measure. As a result, the axilla and the position information of the axilla measured by the position sensor are associated with each other, and the nipple and the position information of the nipple measured by the position sensor are associated with each other. Then, the operator of the ultrasonic diagnostic apparatus 20 abuts the ultrasonic probe 21 on the rib body and measures the position with the position sensor. As a result, the rib body and the position information of the rib body measured by the position sensor are associated with each other. Then, the processing circuit 24h uses the position information of the axilla, the position information of the nipple, and the position information of the rib body to determine a circle centered on the nipple and having a radius of the distance between the nipple and the rib body. Further, the processing circuit 24h draws two tangents from the axilla to the circle. As a result, the processing circuit 24h generates a schematic diagram of the breast shown in FIG. 11A.

Subsequently, the operator of the ultrasonic diagnostic apparatus 20 abuts the ultrasonic probe 21 on the region of interest and measures the position with the position sensor. As a result, the region of interest and the position information of the region of interest measured by the position sensor are associated with each other. Then, the processing circuit 24h obtains the position of the region of interest in the schematic diagram of the breast based on the position information measured by the position sensor of the ultrasonic probe 21. That is, the processing circuit 24h generates scanning position information in which the position of the obtained region of interest is added to the schematic diagram of the breast generated by determining the position to be a landmark based on the position information measured by the position sensor. To do. Then, the processing circuit 24h stores the generated scanning position information in the storage circuit 24f. That is, the storage circuit 24f stores the scanning position information generated by the processing circuit 24h.

In such a case, the position specifying function 35d obtains the position information of the region of interest on the three-dimensional image from the position of the region of interest measured by the position sensor of the ultrasonic probe 21. For example, the position specifying function 35d calculates the ratio indicating the position of the region of interest in the same manner as in steps S1 and S2. That is, as shown in FIG. 11B, the position specifying function 35d obtains the ratio of the positions of the regions of interest in the line extending in the body axis direction passing through the regions of interest based on the measured positions of the regions of interest. Further, as shown in FIG. 11B, the position specifying function 35d obtains the ratio of the positions of the regions of interest on the line passing through the papilla and perpendicular to the body axis. The position information generation function 35e uses the calculated ratio to generate position identification information in the same manner as in steps S3 and S4.

Next, in FIGS. 12A and 12B, the case where the axilla, the papilla, and the end of the clavicle are used as landmarks will be described. As shown in FIG. 12A, the operator of the ultrasonic diagnostic apparatus 20 abuts the ultrasonic probe 21 on the axillary fossa and measures the position with the position sensor, and then abuts the ultrasonic probe 21 on the papilla and positions the position with the position sensor. To measure. As a result, the axilla and the position information of the axilla measured by the position sensor are associated with each other, and the nipple and the position information of the nipple measured by the position sensor are associated with each other. Then, the operator of the ultrasonic diagnostic apparatus 20 abuts the ultrasonic probe 21 on the end of the clavicle and measures the position with the position sensor. As a result, the end of the clavicle and the position information of the end of the clavicle measured by the position sensor are associated with each other. Then, the processing circuit 24h draws a square having the distance between the axilla and the end of the clavicle as one side using the position information of the axilla, the position information of the papilla, and the position information of the end of the clavicle, and is shown in FIG. 12A. Generate a schematic diagram of the breast.

Subsequently, the operator of the ultrasonic diagnostic apparatus 20 abuts the ultrasonic probe 21 on the region of interest and measures the position with the position sensor. As a result, the region of interest and the position information of the region of interest measured by the position sensor are associated with each other. Then, the processing circuit 24h obtains the position of the region of interest in the schematic diagram of the breast based on the position information measured by the position sensor of the ultrasonic probe 21. That is, the processing circuit 24h generates scanning position information in which the position of the obtained region of interest is added to the schematic diagram of the breast generated by determining the position to be a landmark based on the position information measured by the position sensor. To do. Then, the processing circuit 24h stores the generated scanning position information in the storage circuit 24f. That is, the storage circuit 24f stores the scanning position information generated by the processing circuit 24h.

In such a case, the position specifying function 35d obtains the position information of the region of interest on the three-dimensional image from the position of the region of interest measured by the position sensor of the ultrasonic probe 21. For example, the position specifying function 35d calculates the ratio indicating the position of the region of interest in the same manner as in steps S1 and S2. That is, as shown in FIG. 12B, the position specifying function 35d obtains the ratio of the positions of the regions of interest in the line extending in the body axis direction passing through the regions of interest based on the measured positions of the regions of interest. Further, as shown in FIG. 12B, the position specifying function 35d obtains the ratio of the positions of the regions of interest on the line passing through the nipple and perpendicular to the body axis. The position information generation function 35e uses the calculated ratio to generate position identification information in the same manner as in steps S3 and S4.

Further, the position information generation function 35e may further generate depth information indicating the position where the region of interest exists in the cross section including the region of interest in the position in the depth direction with respect to the body axis direction from the nipple. In such a case, the position specifying function 35d specifies the relative position where the region of interest exists in the cross section including the region of interest at the position in the depth direction from the nipple to the body axis direction. FIG. 13 is a diagram for explaining the first embodiment. The right figure of FIG. 13 shows an ultrasonic image. The positioning function 35d divides the breast into six parts along the direction from the nipple to the chest wall, for example, as shown in the right figure of FIG. Then, the position specifying function 35d identifies that the region of interest is located in a region approximately third from the papilla in the region divided into six. The position specifying function 35d passes the position of the specified area of interest in the depth direction to the position information generation function 35e.

Subsequently, the position information generation function 35e generates depth information based on the position in the depth direction specified by the position identification function 35d. For example, the position information generation function 35e further generates an output image based on the position in the specified depth direction. The left figure of FIG. 13 shows a CC tomosynthesis image. The position information generation function 35e divides the CC tomosynthesis image into six along the direction from the papilla to the chest wall, for example, as shown in the left figure of FIG. Then, the position information generation function 35e generates, for example, "3/6 quantile" as depth information indicating that the region of interest is located in the approximately third region from the papilla of the 6-divided region. The position information generation function 35e stores the generated depth information in the storage circuit 34.

Return to FIG. The transmission function 35f transmits the position identification information generated by the position information generation function 35e to at least one of the ultrasonic diagnostic apparatus 20 and the image display apparatus 40 in response to an instruction from the operator. Specifically, the transmission function 35f receives a position identification information transmission instruction from the operator of the image processing device 30 or the ultrasonic diagnostic device 20 via the input interface 31. Here, the transmission function 35f receives an instruction to transmit the position identification information by designating the identification information that can uniquely identify the subject. Then, when the transmission function 35f receives the transmission instruction of the position identification information, the transmission function 35f acquires the position identification information corresponding to the identification information designated by the operator from the storage circuit 34, and obtains the position identification information from the storage circuit 34, and the ultrasonic diagnostic apparatus 20 and the image display device 40. Send to at least one of. Here, when the position identification information corresponding to the identification information designated in the storage circuit 34 is not stored, the transmission function 35f instructs the position identification function 35d and the position information generation function 35e to display FIG. The process of generating the position identification information shown in FIG. 10 is executed. In this way, the transmission function 35f causes the display referred to during ultrasonic scanning to output the position identification information.

FIG. 14 is a diagram for explaining the first embodiment. FIG. 14 describes a case where the scanning position information is displayed on the display of the image display device 40 referred to during ultrasonic scanning and the position identification information is displayed on the display 32 of the image processing device 30. In addition, in FIG. 14, when the operator of the ultrasonic diagnostic apparatus 20 sets an area of interest in an ultrasonic image and scan position information is generated during ultrasonic scanning, it corresponds to the area of interest of the ultrasonic image. An example of referring to the tomosynthesis image and the position identification information will be described.

As shown in FIG. 14, the ultrasonic image during ultrasonic scanning is displayed on the display of the image display device 40 (the ultrasonic diagnostic apparatus 20 side in FIG. 14). Then, when the region of interest is set in this ultrasonic image, scanning position information is displayed on the lower left side of the ultrasonic image, as shown in FIG. In the example shown in FIG. 14, the scanning position information is displayed with a mark indicating the relative position of the region of interest on the schematic diagram of the right breast.

Further, in the example shown in FIG. 14, the display 32 (tomosynthesis side in FIG. 14) of the image processing device 30 is arranged adjacent to the display of the image display device 40. Here, when the operator of the ultrasonic diagnostic apparatus 20 requests the image processing apparatus 30 to transmit the position identification information, as shown in FIG. 14, a CC tomosynthesis image is displayed on the display 32 of the image processing apparatus 30 as the position identification information. Is displayed. The position identification information is a CC tomosynthesis image specified as a cross section corresponding to a region of interest set in the ultrasonic image displayed on the display of the image display device 40. That is, the CC tomosynthesis image as the position identification information displayed on the display 32 of the image processing device 30 corresponds to the scanning position information displayed on the display of the image display device 40. For example, c: d shown in FIG. 9C may be displayed as the position identification information.

Further, the image processing device 30 further displays the depth information on the display 32 of the image processing device 30. For example, in the N-divided breast, depth information indicating the region of interest from the papillary side is displayed. Here, for example, when "2/5 quantile" is displayed as the depth information, the operator of the ultrasonic diagnostic apparatus 20 divides the CC tomosynthesis image into five by referring to the depth information. It can be grasped that the region of interest exists in the second region from the papillary side in the breast. As a result, the operator of the ultrasonic diagnostic apparatus 20 can grasp the position in the depth direction corresponding to the region of interest set by the ultrasonic scanning in the CC tomosynthesis image.

Further, the position information generation function 35e may further generate cross-section information for specifying the cross-section including the region of interest from the cross-section included on the three-dimensional image. In such a case, the position specifying function 35d identifies the cross section including the region of interest from the cross section included on the three-dimensional image. Then, the position information generation function 35e further generates an output image based on the specified cross section. For example, the position information generation function 35e generates cross-section information including a scale and an arrow. In the example shown in FIG. 14, a scale and an arrow indicating the position of the cross section being displayed on the display 32 of the image processing apparatus 30 are displayed. In this scale, the upper side indicates the Head direction and the lower side indicates the Foot direction. The arrow indicates the position of the cross section of the tomosynthesis image displayed on the display 32 of the image processing device 30 when the three-dimensional image is sliced in the body axis direction. This arrow can be moved up and down along the scale. When the arrow is moved, the display 32 of the image processing device 30 may display a tomosynthesis image corresponding to the cross section indicated by the moved arrow.

Further, a plurality of regions of interest may be set in the scanning position information. In such a case, the position specifying function 35d specifies the position of each of the plurality of areas of interest from the scanning position information. Then, the position information generation function 35e generates position identification information associated with the position information of the area of interest on the tomosynthesis image of the cross section specified for each area of interest. Further, when a plurality of regions of interest are set in the scanning position information, the image processing device 30 may display the position identification information corresponding to the region of interest specified in the scanning position information on the display 32. The position-specific information corresponding to each region of interest may be switched and displayed at predetermined intervals.

Although FIG. 14 has described a case where the scanning position information is displayed on the display of the image display device 40 and the tomosynthesis image is displayed as the position identification information on the display 32 of the image processing device 30, the embodiment is limited to this. It's not something. For example, when the image display device 40 has a plurality of displays, the scanning position information may be displayed on one display, and the tomosynthesis image may be displayed on the other display as the position identification information.

As described above, in the first embodiment, the position of the region of interest when the breast of the subject is scanned using the ultrasonic probe is determined from the scanning position information associated with the schematic diagram of the breast. To identify. Subsequently, position-specific information is generated in which the position information of the region of interest is associated with the three-dimensional image generated by irradiating the X-ray tube 15a with X-rays and imaging the breast of the subject from different directions. Then, the position identification information is output to, for example, the display 32. Thereby, according to the first embodiment, for example, it is possible to associate the region of interest at the time of UL imaging with the MLO or CC image of the tomosynthesis image. As a result, the operator of the ultrasonic diagnostic apparatus 20 can refer to the tomosynthesis image corresponding to the region of interest set in the ultrasonic image, and the workflow can be streamlined. Further, since the imaging position by the ultrasonic diagnostic apparatus 20 can be associated with the tomosynthesis image by an objective method, the reproducibility can be improved.

It should be noted that the plurality of components in FIG. 7 may be integrated into one processor to realize the function. The word "processor" used in the above description is, for example, a CPUCentral Processing Unit (CPU), a GPU (Graphics Processing Unit), an integrated circuit for a specific application (ASIC), or a programmable logic device (for example, a simple programmable device). It means a circuit such as a logical device (Simple Programmable Logic Device: SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA). The processor realizes the function by reading and executing the program stored in the storage circuit 34. Instead of storing the program in the storage circuit 34, the program may be directly incorporated in the circuit of the processor. In this case, the processor realizes the function by reading and executing the program embedded in the circuit. It should be noted that each processor of the present embodiment is not limited to the case where each processor is configured as a single circuit, and a plurality of independent circuits may be combined to form one processor to realize its function. Good.

Further, in the first embodiment, when a request for position identification information is received from the operator of the ultrasonic diagnostic apparatus 20 during ultrasonic scanning, position identification information is generated and the generated position identification information is ultrasonically scanned. Although the case of displaying on the display 32 of the image processing apparatus 30 sometimes referred to is shown, the embodiment is not limited to this. For example, after the ultrasonic scanning is completed, the position identification information may be generated when the mammography image and the ultrasonic image are used in combination for interpretation. As a result, the image interpreter can easily observe the tomosynthesis image corresponding to the region of interest set in the ultrasonic image, and the time required for image interpretation can be shortened.

(Second embodiment)
In the first embodiment, the case where the position identification information is generated from the scanning position information has been described. By the way, it may be possible to grasp the position of the region of interest set in the tomosynthesis image during scanning by the ultrasonic diagnostic apparatus. Therefore, in the second embodiment, a case where reference information in which the region of interest in the tomosynthesis image and the imaging position by the ultrasonic diagnostic apparatus are associated with each other is further generated will be described.

FIG. 15 is a diagram showing a configuration example of the image processing device 300 according to the second embodiment. As shown in FIG. 15, the image processing device 300 includes an input interface 31, a display 32, a communication control interface 33, a storage circuit 34, and a processing circuit 350. Of the components shown in FIG. 15, when each component has the same function as that shown in FIG. 7, the same reference numerals are given and detailed description thereof will be omitted.

The processing circuit 350 controls the operation of the image processing device 300. As shown in FIG. 15, the processing circuit 350 includes an image data acquisition function 35a, a finding information creation function 35b, a display control function 35c, an area setting function 351 and a reference information generation function 352, and a position identification function 35d. , The position information generation function 35e and the transmission function 35f are executed. Here, for example, an image data acquisition function 35a, which is a component of the processing circuit 350 shown in FIG. 15, a finding information creation function 35b, a display control function 35c, an area setting function 351 and a reference information generation function 352. Each processing function executed by the transmission function 35f is recorded in the storage circuit 34 in the form of a program that can be executed by a computer. The processing circuit 350 is a processor that realizes a function corresponding to each program by reading each program from the storage circuit 34 and executing the program. In other words, the processing circuit 350 in the state where each program is read has each function shown in the processing circuit 350 of FIG.

The region setting function 351 sets a region of interest in a three-dimensional image generated by irradiating X-rays from an X-ray tube 15a and imaging the breast of a subject from different directions. Specifically, the area setting function 351 is an operation of designating a range of an arbitrary size at an arbitrary position on the tomosynthesis image arranged on the reference screen displayed by the display control function 35c via the input interface 31. Is accepted from the operator. Then, the area setting function 351 sets the range specified by the operator as the area of interest.

For example, the region setting function 351 may automatically detect a candidate region of a lesion from a mammography image using a computer-aided diagnosis function, and set the detected region as a region of interest. In other words, the area setting function 351 sets the area of interest based on the result of the computer-aided diagnosis. Further, for example, the area setting function 351 may accept an operation of selecting an area from the detected areas for the area detected by CAD from the operator and set the selected area as the area of interest. Good.

The reference information generation function 352 generates reference information in which the position information of the region of interest is associated with the schematic diagram of the breast. For example, the reference information generation function 352 generates reference information in which position information indicating a relative position of a region of interest in a three-dimensional image is associated with a schematic diagram of the breast. Here, the reference information generation function 352 specifies a cross section including the region of interest in the three-dimensional image, and uses the relative position of the cross section in the three-dimensional image and the relative position of the region of interest in the cross section as position information. ..

Subsequently, the processing operation of the reference information generation function 352 will be described with reference to FIGS. 16 to 19. FIG. 16 is a flowchart showing a processing procedure by the image processing apparatus 30 according to the second embodiment, and FIGS. 17A to 17C are diagrams for explaining the second embodiment. FIG. 18 is a flowchart showing a processing procedure by the image processing apparatus 30 according to the second embodiment, and FIG. 19 is a diagram for explaining the second embodiment. Further, FIGS. 16 and 17A to 17C describe a case where a CC tomosynthesis image is used, and FIGS. 18 and 19 describe a case where an MLO tomosynthesis image is used.

First, a case where a CC tomosynthesis image is used will be described. As shown in FIG. 16, in the reference information generation function 352, the ratio of the intersection of the cross section including the region of interest of the tomosynthesis image and the straight line connecting the compression plate 14 and the X-ray detector 16a at the shortest distance (a: b). Is calculated (step S101). FIG. 17A shows a case where the breast of the subject is compressed and fixed between the compression plate 14 and the X-ray detector 16a. Each cross section of the tomosynthesis image generated when tomosynthesis imaging is performed in this state is shown by a broken line. Further, in the example shown in FIG. 17A, it is assumed that the region of interest is included in the second cross section from the compression plate 14. As shown in FIG. 17A, in the reference information generation function 352, the ratio of the intersection of the straight line connecting the compression plate 14 and the X-ray detector 16a with the shortest distance and the cross section including the region of interest of the tomosynthesis image (a: b). Is calculated.

The reference information generation function 352 detects the boundary and the papilla of the tomosynthesis image including the region of interest (step S102). FIG. 17B shows a CC tomosynthesis image of the right breast. As shown in FIG. 17B, the reference information generation function 352 detects the breast start (end) position and the papilla as the boundary of the tomosynthesis image. Then, as shown in FIG. 17B, the reference information generation function 352 has a straight line l3 that connects the detected straight line l1 passing through the breast start (end) position and parallel to the X axis and the straight line l2 passing through the nipple at the shortest distance. It is set to pass through the region of interest, and the ratio of the positions of the regions of interest (c: d) is calculated (step S103). In FIG. 17B, the region of interest is indicated by a circle.

Subsequently, as shown in FIG. 17C, the reference information generation function 352 passes through the point forming the ratio of c: d on the straight line L1 passing through the nipple and perpendicular to the body axis in the schematic diagram of the breast, and passes through the body axis. A straight line L2 to be parallel is set (step S104). Then, the reference information generation function 352 determines a point forming the ratio of a: b on the straight line L2 (step S105). Note that FIG. 17C shows a schematic view of the right breast.

Next, a case where an MLO tomosynthesis image is used will be described. As shown in FIG. 18, in the reference information generation function 352, the ratio of the intersection of the cross section including the region of interest of the tomosynthesis image and the straight line connecting the compression plate 14 and the X-ray detector 16a at the shortest distance (a: b). Is calculated (step S201). FIG. 19 shows a case where the breast of the subject is compressed and fixed between the compression plate 14 and the X-ray detector 16a. Each cross section of the tomosynthesis image generated when tomosynthesis imaging is performed in this state is shown by a broken line. Further, in the example shown in FIG. 19, it is assumed that the region of interest is included in the second cross section from the compression plate 14. As shown in FIG. 19, in the reference information generation function 352, the ratio of the intersection of the straight line connecting the compression plate 14 and the X-ray detector 16a with the shortest distance and the cross section including the region of interest of the tomosynthesis image (a: b). Is calculated.

The reference information generation function 352 detects the boundary and the papilla of the tomosynthesis image including the region of interest (step S202). FIG. 19 shows a CC tomosynthesis image of the right breast. As shown in FIG. 19, the reference information generation function 352 detects the breast start (end) position and the papilla as boundaries of the tomosynthesis image. Then, as shown in FIG. 19, the reference information generation function 352 connects the straight line l1 projected on the X-ray tube 15a side through the detected breast start (end) position and the straight line l2 passing through the nipple at the shortest distance. The straight line l3 is set to pass through the region of interest, and the ratio of the positions of the regions of interest (c: d) is calculated (step S203).

Subsequently, as shown in FIG. 19, the reference information generation function 352 generates a straight line L1 connecting the intersections of the schematic diagram projected on the X-ray tube 15a side from the region of interest (step S204). Then, the reference information generation function 352 calculates the points forming the ratio of a: b on the straight line L1 (step S205). Note that FIG. 19 shows a schematic view of the right breast.

The reference information generation function 352 generates reference information as image data in a format such as DICOM format, JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), or bitmap. Here, the reference information generation function 352 generates reference information further associated with identification information that can uniquely identify the subject, and stores the reference information in the storage circuit 34.

Further, the reference information generation function 352 may further generate depth information indicating the position where the region of interest exists in the position in the depth direction with respect to the body axis direction from the nipple. In such a case, the reference information generation function 352 may indicate the relative position of the region of interest at a position in the depth direction from the papillary side to the chest wall direction, for example. For example, the reference information generation function 352 may generate "3/6 quantile" as depth information when the region of interest exists in the third region from the nipple side in the 6-divided breast. The reference information generation function 352 further associates the generated depth information with identification information that can uniquely identify the subject and stores it in the storage circuit 34.

Further, the transmission function 35f according to the second embodiment transfers the reference information generated by the reference information generation function 352 to at least one of the ultrasonic diagnostic apparatus 20 and the image display apparatus 40 in response to an instruction from the operator. Send to. Specifically, the transmission function 35f receives a transmission instruction of reference information from the operator of the image processing device 30 or the ultrasonic diagnostic device 20 via the input interface 31. Here, the transmission function 35f receives an instruction to transmit reference information by designating identification information that can uniquely identify the subject. Then, when the transmission function 35f receives the transmission instruction of the reference information, the transmission function 35f acquires the reference information corresponding to the identification information designated by the operator from the storage circuit 34, and at least the ultrasonic diagnostic apparatus 20 and the image display apparatus 40. Send to either one. In other words, the transmission function 35f causes the display referenced during ultrasonic scanning to output reference information.

Further, the reference information generation function 352 may generate reference information in a form according to the properties of the region of interest. For example, the reference information generation function 352 not only indicates the position information of the region of interest on the schematic diagram when generating the reference information using the template of the schematic diagram, but also the region where the mammary gland density is higher than a predetermined value, or The calcified region, the tumor region, and the like may be extracted from the mammography image and displayed on the schematic diagram. For example, the reference information generation function 352 displays the parts corresponding to these areas on the schematic diagram in a color different from the color of the schematic diagram for each type of area, or marks determined in advance for each type of area. Or display.

Then, the processing circuit 350 according to the second embodiment may compare the reference information with the scanning position information. Here, in the processing circuit 350, for example, when there is an area of interest set at the time of ultrasonic scanning near the area of interest set for the tomosynthesis image, the area of interest set for the tomosynthesis image becomes super The operator at the time of ultrasonic scanning may be notified that the image exists in the vicinity of the region of interest set at the time of ultrasonic scanning. Alternatively, in the processing circuit 350, when the interpretation compares the tomosynthesis image and the ultrasonic image and interprets the image, the region of interest set for the tomosynthesis image and the region of interest set during ultrasonic scanning are close to each other. You may notify the tomosynthesis to do so.

(Other embodiments)
The embodiment is not limited to the above-described embodiment.

In the above-described embodiment, the image processing device 30 or the image processing device 300 has been described as executing the position specifying function 35d and the position information generation function 35e, but the embodiment is not limited to this. For example, the ultrasonic diagnostic apparatus 20 may execute the position specifying function 35d, and the image processing apparatus 30 or the image processing apparatus 300 may execute the position information generation function 35e. Alternatively, the ultrasonic diagnostic apparatus 20 may execute the position identification function 35d, and the mammography apparatus 10 may execute the position information generation function 35e. Alternatively, the ultrasonic diagnostic apparatus 20 may execute the position identification function 35d and the position information generation function 35e. Alternatively, the mammography apparatus 10 may execute the position identification function 35d and the position information generation function 35e.

Further, in the above-described embodiment, the processing circuit 24h of the ultrasonic diagnostic apparatus 20 has been described as generating scanning position information, but the embodiment is not limited to this. For example, the mammography apparatus 10, the image processing apparatus 30, or the image processing apparatus 300 may generate scanning position information.

Further, in the above-described embodiment, the image processing apparatus 300 has been described as executing the area setting function 351 and the reference information generation function 352, but the embodiment is not limited to this. For example, the image processing device 300 may execute the area setting function 351 and the ultrasonic diagnostic device 20 may execute the reference information generation function 352. Alternatively, the mammography apparatus 10 may execute the area setting function 351 and the ultrasonic diagnostic apparatus 20 may execute the reference information generation function 352. Further, the mammography apparatus 10 may execute the area setting function 351 and the reference information generation function 352. Alternatively, the ultrasonic diagnostic apparatus 20 may execute the area setting function 351 and the reference information generation function 352.

In the above description of the embodiment, each component of each of the illustrated devices is a functional concept and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically distributed in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Further, each processing function performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

Further, the control method described in the above embodiment can be realized by executing a control program prepared in advance on a computer such as a personal computer or a workstation. This control program can be distributed via a network such as the Internet. Further, this control program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and being read from the recording medium by the computer.

According to at least one embodiment described above, the workflow can be streamlined in the examination in which the tomosynthesis image and the ultrasonic image are used in combination.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

10 Mammography device 20 Ultrasonic diagnostic device 30 Image processing device 35 Processing circuit 35d Position identification function 35e Position information generation function 35f Transmission function 40 Image display device 100 Medical information processing device

Claims (10)

The breast showing the position of the circle centered on the nipple and the axilla, which was generated based on the position information measured by the position sensor of the ultrasonic probe when the breast of the subject was scanned using the ultrasonic probe. The first storage unit that stores the schematic diagram of
A processing unit that obtains the position of the region of interest in the schematic diagram of the breast, and
A second storage unit for storing a position of the region of interest in schematic view before Kichichi tufts,
A third storage unit that stores a tomosynthesis image, which is a three-dimensional image of the breast acquired by a mammography apparatus, and a third storage unit.
Based on the position of the region of interest in the schematic diagram of the breast, two intersections of the first straight line passing through the region of interest and parallel to the body axis direction or the X-ray tube direction and the circle are obtained. A calculation unit that obtains the ratio of the distance between each of the two intersections and the region of interest, and uses the ratio to determine the cross section including the region of interest on the tomosynthesis image.
A medical information processing apparatus including an image generation processing unit that generates an image of a cross section determined on the tomosynthesis image. The second storage unit stores the position of the region of interest in the schematic diagram of the breast as scanning position information.
The arithmetic unit, based on the scanning position information, determines the cross section including the area of interest on said tomosynthesis images,
The image generation processing unit generates the association was images the positional information of the region of interest to the cross-section is determined on the tomosynthesis images, medical information processing apparatus according to claim 1. Wherein the processing unit generates a scanning position information denoted by the position before Symbol ROI in the schematic view before Symbol breast,
The medical information processing apparatus according to claim 1 or 2, wherein the second storage unit stores the scanning position information generated by the processing unit. The calculation unit obtains the positional information of the region of interest before Symbol tomosynthesis section determined in the image,
The image generation processing unit, the generating the images that associates position information of the region of interest on the cross, medical information processing apparatus according to any one of claims 1 to 3. The arithmetic unit further wherein obtain the intersection of the schematic diagram the nipple and as body-axis direction perpendicular second straight line in said first straight line, between the said intersection the respective circle and the nipple The medical information processing apparatus according to claim 4, wherein a ratio of distances is obtained, and the position information of the region of interest in the cross section is obtained using the ratio. The arithmetic unit, the relative position of the region of interest is present in the cross section including the region of interest, specified by the position in the depth direction with respect to the body axis direction from said teats,
The medical information processing device according to any one of claims 1 to 5, wherein the image generation processing unit further generates depth information indicating a specified position in the depth direction. Before Symbol image generation processing unit further generates a cross-section information indicating the position of the pre-Symbol cross, medical information processing apparatus according to any one of claims 1 to 6. The second storage unit stores the identification information that can uniquely identify the subject in association with the scanning position information in which the position of the region of interest is attached to the schematic diagram of the breast.
The calculation unit obtains the scanning position information corresponding to the specified the identification information from the second storage unit, based on the scanning position information, determines the cross section including the region of interest on said tomosynthesis images to that, the medical image processing apparatus according to any one of claims 1-7. The image generation processing unit includes front Kiga and images, it is stored in the second storage section in association with the scanning position information used to generate an equivalent該画images, medical according to claim 8 Information processing device. The medical information processing apparatus according to claim 8 or 9, wherein the second storage unit stores the scanning position information in a DICOM format or a JPEG format.

Images